1

Help! Creating clone from a cosmic installed bootable USB stick;

I have a working running USB stick with lubuntu 18.10 cosmic on it. I installed the stick by a normal lubuntu 18.10 .iso.

Now I learned, or always understood that cloning a linux system is easy. Today and yesterday I learned in practical experience it is not. Result is a not working 2th usb stick and even partition manager doesn't like it.

What I do:

How do I use dd to clone an external USB drive installation to a local hard disk?

I use that syntax of the dd command to clone my working linux installed bootable usb stick to another usb stick. I do this by using a third lubuntu installation. In other words, I do not try to backup an active running lubuntu session itself. Yesterday I tried backingup from the running system , it didn't work. Than I found topics stating that this method doesn't result in a readable usb clone, so today I decided to use usb stick number 3 which is starting an external Linux system so I can clone usb stick1 to usb stick number 2.

Complex? Maybe, but I wanted to be sure everything is tested out. And using this context:

sudo dd if=/dev/sdx of=/dev/sdy bs=8M && sync

Doesn't work at all. Result: It creates a clone usb stick. Whne I try to run it, the grub comes up, I can select lubuntu, I see it starting ,the . . . . dots come on the screen which creates hope. Than The system falls back in a initdos or puppydos prompt (or whatever it is called,) I tried to do a fsck command on the patition, rebooting, didn't work. In fact it seems to hang on a video driver or something. But how is that possible if the system is supposed to be an exact clone of the working usb? I can run mc midnight commander, when I type it in the prompt screen (I installed mc before backing up at the system of the original usb stick).

To complete my story I must add, that I also tried to dd backup first to the internal harddrive, before restoring it on usb stick 2. Didn't work either.....

Would there by anybody out there, whom found a way to do what I want to do for a lubuntu cosmic 18.10 USB running sytem?....

I know there are tools like clonezilla and stuff, but doing what I tried should work. Why doesn't it? Does dd only work when you restore the backup on the same usb stick? (note that both usb devices I used are not of the same brand, but have the same size, 32gb....)

Some help required.

Thanks!

update: i forgot to mention 2 more things:

1.On the USB stick wih my working install, I have 2 partitions. A ext4 / system partition and a small 6 gb NTFS partition I use for sharing files. I tried dd if=dev/sdd and if=dev/sdd1 , same end result.

2.The installation of the original usb stick is based on a UEFI partition table! The bios in my laptop is set on UEFI as well. Just in case it is important...

leeuwtje
  • 231
  • 3
  • 12

1 Answers1

2

Cloning from one drive to another drive

What you describe in your question works in general.

sudo dd if=/dev/sdx of=/dev/sdy bs=4096 && sync

I have done it successfully with USB pendrives as well as with HDDs and SSDs.

But there are several things that can make it go wrong.

  • The target drive must be at least as big as the source drive. Not one single byte smaller. It is not enough that the drives have the same nominal size, for example 16 GB. You can check with

    sudo parted /dev/sdx u b p
sudo parted /dev/sdy u b p

    where x and y should be replaced with the device letters for the source and target drives, and u b makes the output in bytes and p tells parted to print (to standard output).

  • If there is a GUID partition table, GPT, (and not an old style MSDOS partition table), and the drives have not exactly the same size, you must repair the backup partition table at the tail end of the drive. The previous command lines with parted will also show the partition table. You can use gdisk for this repair job.

  • If the physical sector sizes are different between the source and target drives, there will also be problems. Small and old drives have usually 512 bytes physical sectors, but some new drives (medium size and huge size) sometimes have 4096 bytes physical sectors. This can cause problems after cloning, and it is probably better the install a fresh system in the new drive and transfer the files, that you want into the new drive (copy on the file level instead of cloning on the drive level). The previous command lines with parted will also show the physical sector size.

  • Both the source drive and the target drive must be healthy. If there is a bad sector (physical sector), the corresponding data cannot be transferred for that particular location, and it may cause a problem (big or small, depending on what is stored in that location).

  • Please do not clone the drive that you have booted from. For example, boot from a Lubuntu live drive made from a Lubuntu iso file in order to clone from a drive with an installed Lubuntu system to some other drive.

    No partition on the source drive and the target drive should be mounted. Unmount if some partition is mounted.

    sudo umount /dev/sdxn

    where x is the device letter and n is the partition number.

  • Cloning with dd is risky, because it does what you tell it to do without any questions. A minor typing error is enough to make it overwrite the family pictures. For this reason it is better to use a tool which helps you identify the target drive and has a final checkpoint, where you can double-check that things will go right.

  • You can also clone with Clonezilla. Boot a USB pendrive or DVD disk made from a stable clonezilla.iso file. This is safer than dd because of several checkpoints and questions, and faster because Clonezilla is smart enough to only copy used blocks (and skip free blocks) of the file systems. This makes a big difference if a file system is far from full.

  • And last but not least, you must let the process finish properly and let the system flush the buffers to the target drive. This is why it is important to run sync and wait until it finishes, and the terminal window returns to prompt.

Cloning via an image file

  • What is described here is also relevant if you clone in an indirect way by creating an image file from the source drive and restore from the image file to the target drive.

After the cloning, connect only one drive to the computer

  • If there are writable file systems, you must not boot a computer with both the source and target (of the cloning) connected. These drives have identical data, and there will probably be confusion, which can destroy one or both of the drives.

    So boot into either the original (source) drive or into the cloned copy (target) drive, and keep the other drive disconnected (unplugged).

sudodus
  • 45,126
  • 5
  • 87
  • 151
  • Wow, thanks sudodus, for spending so much time for a technical deep explanation of what actually is happening in the dd process and a collection of problems that can occur cloning usb sticks with it. Ok. I will see what I can do with mkusb to get what I wanted. Could you describe the exact steps in the terminal prompt for the mkusb route with sync? – leeuwtje Dec 02 '18 at 14:13
  • One more thing, about the different kind of usb drives where you refer to old and new drives. Would this also give problems in case of trying to clone a compact flash card to an new usb stick? – leeuwtje Dec 02 '18 at 14:16
  • 0. Please do **not** clone the drive that you have booted from. For example, boot from a Lubuntu live drive made from a Lubuntu iso file, identify the source drive (for example with `sudo lsblk -f`and `sudo lsblk -m`); 1. Install mkusb according to the link in my answer; 2.Start mkusb-dus from the menu in Lubuntu or from the command line in a terminal window `dus ` for example `dus /dev/sdx` where `/dev/sdx` is the source drive. mkusb (dus) will help you identify the target drive and run `sync` etc automatically. – sudodus Dec 02 '18 at 15:19
  • 3. I think it will work to clone from a compact flash card to an new usb stick. – sudodus Dec 02 '18 at 15:31
  • 4. Another important thing, that might cause problems: No partition on the source drive and the target drive should be mounted. (I added it to the list in the answer.) – sudodus Dec 02 '18 at 15:37
  • I tried many attempts yesterday, but the results is the same. System boots but breaks before the login screen comes up. leaving me with a prompt. Since my destination usb was indeed larger mkusb gave an error. Afterwards I tried to backup only the system root by dev/sdd2 to an .iso file and clone that with mkusb, didn't work. I gave up for now. Thanks anyway. – leeuwtje Dec 03 '18 at 10:44
  • Too bad you have these problems. If/when you feel ready to try again, you can write a comment and include @sudodus, which sends an alert to me. It would help if you connect the drives that are involved in this cloning effort and run the commands `sudo lsblk -f` and `sudo lsblk -m` and `sudo parted -ls`. Edit your original question to show the output of these commands. Indent each line 4 spaces in order to render the output as 'code'. It will make it easier for me to understand the partition structure and to help you. – sudodus Dec 03 '18 at 11:15
  • 1
    @sudodus you don't need to "Indent each line 4 spaces in order to render the output as 'code'". Just paste in the text, select that text, then hit the `{}` icon which will format it into human-readable text. If you have "preview" enabled, you can see how your question/answer will look to others. – heynnema Dec 04 '18 at 14:50
  • @heynnema, Yes I know, but sometimes it is easier to suggest the basic way to do something. Furthermore, we should add that if the preceding text is indented, the text must be indented another 4 spaces etc to be rendered as 'code' :-) – sudodus Dec 04 '18 at 15:41
  • @sudodus, last weekend I tried a different approach, installing a new stick from scratch to avoid the cloning errors of before. Now something interesting has happened. After I migrated all my stuff from the old stick to the new running lubuntu stick (/home/profile , apt-clone etc), I ran boot repair to have all sticks running in my UEFI setup. Afterwards, a new situation. MY new stick runs perfect, but now the exact same problem is on the old stick. It boots, but stops with a prompt ,refuses to start LXQT desktop. But nothing has changed on this system! It is a healthy setup? – leeuwtje Dec 10 '18 at 11:40
  • @sudodus. So now I start thinking, maybe this wasn't a DD clone problem at all, cause since now I have setup a new lubuntu stick by using the installer, the new sticks works great after migrating all my stuff, and now the old one suddenly stops booting to desktop with the same errors that came up , during my attempts to boot up the dd cloned stick before. So, I think something else is going on. There is also a new question I have now. My stick only runs on this laptop. By the EFI bootmanager. – leeuwtje Dec 10 '18 at 11:46
  • @sudodus But I would like it to run at any UEFI system, like a ubuntu live usb stick. How do I configure advanced options in boot-repair to get that? – leeuwtje Dec 10 '18 at 11:46
  • @leeuwtje, I don't know what stopped your old USB stick from booting correctly, but something has changed, either in the UEFI/BIOS system or in the stick itself, even if it is nothing that you intended or noticed. Installed systems are stable by themselves, but the boot system can be upset easily in UEFI mode (for example by Windows 10). An alternative might be a **persistent live** system. Such a system is not as stable as an installed system, but the booting is more stable and it more portable between computers. You can try with [mkusb](https://help.ubuntu.com/community/mkusb). – sudodus Dec 10 '18 at 13:10