I’ve had time this week to get familiar with our test server environment which is a rather old Dell server running VMWare ESX Server 3.0.1. After creating a new VM and getting a base linux OS configured, I wanted to clone it for later use. I’ve used VMWare Workstation and VMWare Server before and the cloning process there is very simple–just copy. On ESX Server, there are a couple more steps required and then some other steps that are optional, but make life easier in the long run.

Here is the process I followed to perform a clone. It worked well and I was successful in getting my cloned VM running. I didn’t find a process describing exactly what I did, so I thought I’d post it (mostly so I can find it later when I forget what I did). I did find a similar process for a slightly older version and most of this post comes from that process with a few exceptions.

  1. Perform a clean OS shutdown of the guest OS running in the VM. Ideally, this will result in the VM being powered off.
  2. Login to the ESX Server using SSH as the root user.
  3. Use “vdf –h” to determine where the VMFS filesystems are mounted. In our example, this is /vmfs/volumes/AttachedSCSI
  4. Locate the current VM image (at /vmfs/volumes/AttachedSCSI/shelob).
  5. Run “cp –r /vmfs/volumes/AttachedSCSI/shelob /vmfs/volumes/AttachedSCSI/shelob-clone &
  6. Go do something else for a while to let the copy finish (at least 30 mins)
  7. Once the copy is finished, go to the new directory (shelob-clone) and rename all the files to use the new clone name. For example, the string “shelob” will be in the filename for each file. You should rename the file so that “shelob-clone” is in the filename instead of “shelob”.
  8. Once you’ve renamed all the files, run “file *” in this directory (shelob-clone).
  9. Edit all files (back them up first) that are listed as “ASCII text” or “ASCII English text”. In each of those files, you may find references to the old filenames or directory names. Modify the references appropriately to use the new names.
  10. Also edit the shelob-clone.vmx file to remove the following lines:
    ethernet0.generatedAddress = "....."
    ethernet0.generatedAddressOffset = "....."
    uuid.location = "....."
    uuid.bios = "....."
  11. Add the following line to shelob-clone.vmx:
    uuid.action = "keep"
  12. Register the new cloned VM with ESX server by running the following command (this should all be on one line):
    vmware-cmd -s register /vmfs/volumes/AttachedSCSI/shelob-clone/shelob-clone.vmx
  13. Go to the Virtual Infrastructure client and start the new VM. If the guest OS is linux, you should expect kudzu to detect that the old Ethernet controller was removed and a new one has been added. The benefit of this detection is that it will prompt you to add network configuration information. When you’re prompted, you can choose the appropriate IP configuration/address for this new guest VM. Note that you should make sure that your original VM and the new VM have different IP addresses if you intend to have them running at the same time.

Please post suggestions in the comments and I’ll respond and/or update the post.

These steps are going to come in handy as I start playing with the VMs that eventually get posted up at www.OracleVMs.com.

  • Alfred

    Hi, I tried doing this and got the following errors.
    cp: cannot open `Test/Test.vmxf’ for reading: Permission denied
    cp: cannot open `Test/Test.vmsd’ for reading: Permission denied
    cp: cannot open `Test/Test-flat.vmdk’ for reading: Permission denied
    cp: cannot open `Test/Test.vmdk’ for reading: Permission denied
    cp: cannot open `Test/Test.nvram’ for reading: Permission denied

    The VM is stopped. I continued on and edited the vmx file. When I started the VM, it said there is a missing file and stopped. BTW you need to rename the VM inside the vmx file or there will be double entries.

  • Alfred

    Ok I got it. Need to sudo first.

  • http://www.dannorris.com/ dannorris

    Good, let me know if you encounter more issues.

  • dennis young

    i just used the vmware converter tool to clone a image and it worked great.

  • http://www.dannorris.com/ Dan Norris

    I didn’t know the converter tool could do clones. I’ve used the P2V (physical to virtual) converter to create a new VM based on a physical system. Is that the tool you’re talking about?

  • dennis young

    This tool will do physical to virtual conversions too.

    http://www.vmware.com/products/converter/

  • http://www.dannorris.com/ Dan Norris

    A clone as described in my post is to take an existing ESX virtual machine and copy it to create a new ESX virtual machine. The converter will take a physical system and convert it to a VM (but you have to use the enterprise version to convert to an ESX server destination).

    I’ve used the converter free tool and it works quite well, but that’s not what this post was about.

  • dennis young

    you can use the converter tool to take a esx virtual machine and copy it to another esx server, the same esx server or to a vmware server or workstation. i’ve done it multiple times in the last two days.

    the converter tool addresses exactly what this post was about.

  • http://www.dannorris.com/ Dan Norris

    I see now–I think you’re talking about the licensed version of the converter tool (I’ve never used the licensed version). With the free version of the converter tool (they call it the “starter edition”), you can’t do anything related to ESX, so I guess that’s where the post here would come in handy.

    Please correct me if I’m wrong.

    Reference: http://www.vmware.com/products/converter/get.html

  • dennis young

    I used the free version and it let me clone from esx to esx. try it! it’s pretty slow but it works.

  • Casey

    How are your 13 steps better than the one step of:

    “vmkfstools –-i

    ?

  • http://www.dannorris.com/ Dan

    I’m not sure I know what vmkfstools -i does. Looks like it just clones a disk, not a whole VM by reading the docs. This process will clone an entire VM, possibly to a different server if you’d like. Maybe you know more about what vmkfstools -i does–I haven’t tried it myself.

  • Andrzej Z.

    Will this process allow me to restore host to the most recent status? Will it be possible to traverse through snapshots taken on the source host?

  • http://www.dannorris.com/ Dan

    @Andrzej Excellent question. I haven’t tested that personally, but I should expect that all snapshots would be available after importing the VM on the target host. If I have time in the next few weeks, I’ll try it. If you try it and have some results, let me know and I’ll update the post here.

  • Andrzej Z.

    Dan, It looks it works for most recent status of the host as well as the snapshots. Additional steps are required. Beside renaming files you will also need to modify all non-binary files to reflect all references to the new file names.

  • http://www.dannorris.com/ Dan

    That’s what step #9 above is all about–editing the contents of the non-binary files to change the references. Are you saying that you have to do more than what’s stated there in step 9?

  • Andrzej Z.

    :) not really, I just overlooked that. Concluding the entire procedure is complete and it works.

  • Pingback: Confluence: ICT()

  • sycomsimon

    Why not just right click the virtual server that you have shutdown and want to clone in Virtual Centre. Once you right click, choose clone.

  • http://www.dannorris.com/ Dan

    The reason I developed this procedure was because I didn’t have access to Virtual Centre. This is the “manual” way–if you have VC, you’re right, it’s a lot easier. However, last I checked, VC isn’t free, so not everyone will have it configured.

  • http://artisticdigital.com Ryan Lanham

    We own 3 ESX Servers and this is the method that we use ever time to copy our Master Linux Clone. Please keep in mind that if you are copying a striped version of Unix based OS you may need to some extra work to get the NIC/ETH0 online. We use Ubuntu 7.10 LTS and one you clone you must delete the: /etc/udev/rules.d/70-persistent-net.rules and reboot in order for the server to detect the new NIC/ETH0 that it thinks it has.

  • http://www.dannorris.com/ Dan

    @Ryan Thanks for the extra tip!

  • Gavin Hill

    Couldn’t you simply perform the following successfully?

    1. Create a new machine using VM Infrastructure Client choosing the same base OS that you want to clone.
    2. Once the machine is created, you log into the ESX server as root through SSH and locate the new VM machine (VMnew).
    3. Now what you can do is copy the “VMoriginal-flat.vmdk” file that you want to clone and replace the new VM “VMnewflat.vmdk” file with “VMoriginal-flat.vmdk”, making sure you change the name.
    4. Start the new VM from the VM Infrastructure Client.

  • http://www.dannorris.com/ Dan

    @Gavin: That sounds like it might work–I haven’t tried it though. Have you tried it and was it successful? Unfortunately, my ESX environment isn’t available for testing any longer, so I can’t test it out.

  • http://s23.org mutante

    The method proposed by Gavin worked for me with a Debian VM. Like Ryan described above for Ubuntu, here i needed to delete /etc/udev/rules.d/25-persistent-net.rules and reboot to get the NIC back up.

  • http://www.dannorris.com/ Dan Norris

    Thanks for the update and confirmation that the method works in more
    than one specific case.

  • wmz

    If you would like to clone virtual machine on esx server without the help of virtual center here is the instruction to do so

    log into esx server

    su to root and type the following command

    $ vmkfstools -i /vmfs/volumes/datastore/sourcevm/sourcevm.vmdk /vmfs/volumes/datastore/destvm/destvm.vmdk

    this will clone the virtual machine from source to destination

    once you have finished cloning

    go to vi client and run file->new virtual machine

    use custom instead of typical

    give the same name as the destination virtual machine name

    use next next until you get to point where it ask’s you for select disk change it to use an existing virtual disk and specify the path to .vmdk file

    and click next and next and finish this job

    Now you should be able to turn on the machine with out any issues

    Hope this works for you ;)

  • http://www.dannorris.com/ Dan Norris

    That looks relatively easy compared to what I tried to do originally.
    We've upgraded our ESX server now, so I'm not sure if this method works
    on 3.0.1 or not, but next time we need to clone, I'll give it a try on
    our current version.

    Thanks!

  • Tom

    the VM Explorer tool let's you clone or move your vm's from one server to the other, btw for most features it's free
    http://www.trilead.com/Products/VM_Explorer/

  • http://www.dannorris.com/ Dan Norris

    Tom,

    Thanks for the tip. One of the admins here installed the tool to try it out and says it's awesome and can't believe that he gets exactly the types of things he needs for free. There is a pay-for version, but the free version does all he needs, so he's quite pleased. You've made someone's life easier–thanks!

  • Tom

    btw, latest version supports now ESX 3i

  • HercuLeeZ

    Hi Dan,

    I wonder if you can provide me with your (colleagues) backup and restore process with VM Explorer?
    I have been able to backup and restore but, I've not been able to get the restored VM to detect the operating system.

    If you can help, I would very much appreciate it!

    Thanks!
    Herc

  • http://www.dannorris.com/ Dan Norris

    I forwarded your note/question to the admin that's tested it. I'm not sure he's done much with backup and restore, but if he has, he should respond with an email pretty soon.

  • szulfi

    Hi,
    I cloned the source OAS to target using perl scripts provided in clone/bin on OAS. After cloning my source OAS became very slow.

    Please let me know if i need to remove any files after cloning on source?

  • http://www.dannorris.com/ Dan Norris

    I don't think so and there's no reason why I would expect your source to become slow. You can read the clone scripts too and I don't think you'll find anything in there that would be a cause for slowness. There's nothing you have to delete, but if you want to clean up, I think the files that the pre-clone step create are in $OH/clone somewhere.

  • Satbir

    Very helpful article

    Thanks

  • http://www.dannorris.com/ Dan Norris

    Glad you enjoyed it. At this point, I think the comments are possibly more useful than the original article!

  • Jayant

    Good Stuff – good article leading to interesting and very useful comments & thoughts.
    Thank you

  • Pingback: Confluence: Valdis Pukis()

  • Ivan Stoev

    Perfect. The VMware Infrastrcture Client Clone operation kept timing out. You saved my weekend! Thank you.

  • munavar

    this article is very useful and very helpful to me thanks for posting this article. Thanks a lot

  • http://compilefailcry.blogspot.com/ Jose Fernandez

    You can also copy and manually edit the VMX file to reconfigure the new server.

    Also note that if you have snapshots this clone process will not clone the data that came after the snapshots. You have to merge the snapshots first – at least as far as I can tell

  • okgo

    vmware-cmd -s register

    doesn't work on 3.5

  • barcelonaflats

    very good information..thanks for sharing

  • Anonymous

    very good information..thanks for sharing

  • Pingback: Communication()

  • http://www.bluecava.com Chris

    Thanks, this worked perfectly. It is perplexing that they wouldn’t add the clone VM feature in the free version of ESX. Either way, I will just script up this process and call it a day. Thanks again.

    Chris
    BlueCava.com

  • Pingback: Confluence: ICT()