How can I configure the initial data transfer without bogging down the network?


THE INFORMATION IN THIS ARTICLE APPLIES TO:

  • WAFS (All Versions)
  • CDP (All Versions)

QUESTION

How can I configure the initial data transfer without bogging down the network?

ANSWER

IMPORTANT:

Do not use this procedure before discussing your specific requirements with Customer Support.

 

WAFS/CDP native internal transport over your WAN is optimized for the byte-level differences that are transferred for changing files. WAFS/CDP has a maximum bandwidth use of around 4 GB per hour even on the fastest network. You may want to shorten the time for a first large initial full file transfer by using cloning.

 

You can easily setup your target computer's Agents, so that the initial data transfer does not use the network (via WAFS/CDP) to transfer your data to the other location: this is Cloning. If the data is already at the remote site, and is exactly the same as at the source, you can also use cloning to set up the remote site without any data transfer. If the data is not already at the remote sites, you can use movable media, FTP, or any other method to transfer it.

[The standard way to move data is to install Agents on all locations and then connect to the existing Job on them all. ALL of the data will appear in seconds on the second machine. It is immediately accessible and useable, BUT it is still transferring over the network. They stream when you open them so they will open more slowly until the initial transfer finishes.]

Overview

To clone an Agent, finish installing and setting up WAFS/CDP at one site, then send the catalog information that this source Agent has and your data to the target(s) where you will trick the remote Agent into believing it did a native transport. You will need:

  • A copy of the source Agent's AVMF and AVM0 directories.

  • A dump of the source Agent's HKEY_LOCAL_MACHINE\SOFTWARE\WAFS/CDP\WAFS/CDPClient registry.

Once you get these 2 pieces from the source, you can restart the Source Agent and use the data again. Once you get the destinations cloned, they will "see" any changes made in the meantime and update themselves.

Instructions

Before you begin, if they are not already installed, install an Agent on the Source (where your data already is) and install the WAFS/CDP Server.

Follow these steps exactly. 

  1. Configure the Source Agent:
    1. Create new Job.
    2. Replicate a folder or drag files into the WAFS/CDP volume.

    Before continuing, wait for the WAFS/CDP Server and the Source Agent to be in steady state (files 100% completely uploaded to the Server) in the General info tab. This is CRITICAL.

  2. Download and deploy an Agent on the Destination. The Agent on the Destination must be live and connected; the Job, volume name, and data do not matter.
  3. Stop the Destination Agent.
  4. Stop the Source Agent.
  5. Dump the Source Agent's HKEY_LOCAL_MACHINE\SOFTWARE\WAFS/CDP\WAFS/CDPClient registry tree:
    1. At a command prompt, type regedit.
    2. Export the tree from the registry; select the NT4 format when saving).
  6. Copy the exported tree to a CD, tape, hard drive that you will ship, or FTP or email it over to the remote sites where the Agents are located.
  7. While the Source Agent is still shut off, copy its AVM0 and AVMF directories from the Source computer onto your media (or send it to the remote Agent sites).
  8. Perform this step ONLY IF all your files are ALREADY at the remote site (s). Otherwise skip this step. If the exact (meaning 100% exact if you do a CRC) files/folders are at the Destinations by other means, there is NO NEED to transfer them again. Before sending the copy of AVMF and AVMO from the Source, go into these copies and find the "P" directory that has your files (either AVM0 or AVMF will have them - not both). Delete the files from these copies of AVMO or AVMF - you will then have an empty "P" directory. Then send the copy of AVMO and AVMF with the empty P to all the Agent destinations. Once at the destinations, copy the files (that were already at your Destination) back into the now-empty "P" directory in the copy of AVMO or AVMF and proceed. REMEMBER: AVMO and AVMF must now be 100% IDENTICAL at both/all the sites.
  9. NOTE: Make sure that the files at your remote site are identical to the source files at the exact time you took that registry dump. They MUST be exact bit to bit.

  10. Once you get the registry and the AVMO and AVMF copied, restart the Source Agent. Your files can now be used at the Source again. Send your media or email the registry dump and AVM0 and AVMF to the remote sites. The remote sites will "see" that the Source data has changed and will update themselves later.

  11. Once your files/media arrive at the Destination computer, with the Destination Agent still OFF, restore (import) the registry dump onto the Destination computer and place the AVMO and AVMF on the Destination comptuer into the exact same drive letters and locations as it was on the Source (e.g., the Source C:\AVM0 should be copied to the Destination C:\AVM0). If it is not possible to have everything in the identical location, search the copy of the registry file you dumped and modify it so that every instance of the locations are changed to where you want it on the Destination/target.
  12. Make sure you do not already have a folder with the same name and in the same location on the Destination as was used for your files at the Source. The Destination Agent will create it all for you on startup (from the AVMO, AVMF, and registry that you just imported).
  13. Start the Agent on Destination. You are all set - users can use the files here now too.
  14. Note: It is possible that the IP address on the destination Agents is not correct, and the Agents will not connect. This will be because the IP address at the machine you cloned from, used a local LAN address to communicate with the WAFS/CDP server. If the address at the destination Agent is not proper for a remote connection, then you will need to go into the registry at the destination, find the wrong address and edit the registry to the proper IP address of the machine running the WAFS/CDP server.

    Note: As soon as you finished copying all the files, registry, etc. at the Source, you can re-start the Source Agent and use all your data files at the source. If there are changes in the data at the Source since the point in time you copied the data, then the Destination Agent will correctly grab the changes from the Server once it comes up. Nothing will be lost.

    Note: If the files had not completed the upload from the Source Agent to the WAFS/CDP server when you cloned at the Source (on the Agent's General Info tab), then once the clone is done and the Destination Agent comes on line, the Destination Agent will upload data to the Server. You can use the files at the destination during this transfer.