1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Linux What is this Linux thing?

Discussion in 'Software' started by Glider, 27 Jul 2006.

Page 15 of 16
  1. Corky42

    Corky42 Where's walle?

    Joined:
    30 Oct 2012
    Posts:
    9,648
    Likes Received:
    388
    Would be good if i explained what i was trying to achieve, wouldn't it. :)

    Basically I'm getting together the commands needed to copy files off of a USB stick, write an ISO to it, recreate a partition and copy the files back to it. I've ended up with this method because GUI based programs write an ISO9660 partition to the entire 64GB drive leaving it unusable as a storage medium, whereas dd left me with 60 odd GB of unpartitioned space afterwards.

    Because i don't want to do it all by hand and risk forgetting the commands when i need to update the live USB/install medium in years to come i though making script to update the USB when needs be would be better.
    Not even when the file ends with an ".iso" extension, like they do? Either way I've also tried simply using "manjaro*" (without the quotations) and it still claims the file "
    manjaro-kde-18.1.1-191015-linux53.iso" that's in the directory it's being run from and it can find when using the full filename isn't there.
     
    Last edited: 30 Oct 2019
    Corky42, 30 Oct 2019
    #281
  2. Gareth Halfacree

    Gareth Halfacree WIIGII! Lover of bit-tech Administrator Super Moderator Moderator

    Joined:
    4 Dec 2007
    Posts:
    17,130
    Likes Received:
    6,719
    Not even. With "*." you're saying "any string of characters plus any single character," which is redundant unless you're in the very odd scenario of needing to match any string of two or more characters - which you're not.

    You won't be able to do what you're trying to do with dd, 'cos dd doesn't like wildcards (so you don't screw everything up.) What you *can* do is take a different approach - this is Linux, so there's always a dozen ways to do something.

    You could...

    for i in manjaro*iso; do dd if=$i of=/dev/sdb status=progress; done

    That'd do what you're trying to do, but if you've more than one manjaro*iso file it'll write the first, then go back and overwrite it with the second, then go back and overwrite it with the third, and so on and so forth.

    You could...

    find . -name manjaro*iso -exec dd if={} of=/dev/sdb status=progress \;

    But that has the same problem.

    You could...

    for i in $(ls -r manjaro*iso | head -n1); do dd if=$i of=/dev/sdb status=progress; done

    That has the advantage of only doing *one* matching file - the one that's furthest down in the alphabetical sorting order, which means if you have manjaro-2019-08-11.iso and manjaro-2019-10-30.iso it'll write the latter rather than the former.

    In any of these cases, though, you're going to need to be really, really, really careful. Scripting dd is usually a bad idea, 'cos it won't think twice before wiping everything in its path. I'd especially recommend avoiding using /dev/sdX device names as the output file target: there's no guarantee that what was /dev/sdb today will be /dev/sdb tomorrow. If you insist on doing it this way, you're going to want to specify the target via UUID so you don't programmatically nuke something you care about. You can snag 'em via sudo blkid.

    EDIT:
    Or just use Balena Etcher, which is point-'n'-drool. Or right-click on the ISO and choose Disk Image Writer, if your DE has it - and if not, it'll probably have an equivalent.
     
    Gareth Halfacree, 30 Oct 2019
    #282
    Corky42 likes this.
  3. Corky42

    Corky42 Where's walle?

    Joined:
    30 Oct 2012
    Posts:
    9,648
    Likes Received:
    388
    Top man! :)

    I suspected it was something to do with dd not liking wildcards because of the chance of blowing things up, great to have it confirmed as it can be hard to know if it's my own dumbness or a limitation, I'll look into your suggestions, either one will probably do as there will only ever be one manjaro-whatever.iso in the directory the script will be run from.
    Yea, i have to say I'm really enjoying the freedom, i feel like a little kid who's just learnt something new that he want's to share with everyone even though it's probably old hat to them. :)
    I was going to do that but doesn't the UUID change when the partition table is changed/overwritten as would happen when dd'ing a bootable ISO to a USB.
    It does but it's a bit bloated (something like 100Mb) and their collection of data on what images you've used put me right off, plus I'm guessing it will do what most GUI image writing programs do and write a 2-3GB ISO to the entire 64GB drive leaving me with no space to store anything else.
     
    Corky42, 30 Oct 2019
    #283
  4. Gareth Halfacree

    Gareth Halfacree WIIGII! Lover of bit-tech Administrator Super Moderator Moderator

    Joined:
    4 Dec 2007
    Posts:
    17,130
    Likes Received:
    6,719
    Fixed that for you. Always assume the worst!
    You'd guess wrong. It's basically dd with an Electron UI (hence the size.) In fact, neither Etcher nor Disk Image Writer will make any attempt to resize a written image: if you write a 1GB image to a 32GB drive, you'll have a 1GB image on a 32GB drive and around 31GB of free space.
     
    Gareth Halfacree, 30 Oct 2019
    #284
    Corky42 likes this.
  5. Corky42

    Corky42 Where's walle?

    Joined:
    30 Oct 2012
    Posts:
    9,648
    Likes Received:
    388
    Really, I'm going to have to go back and have another look at that then as by the sounds of it i missed something.

    EDIT: quick update, you're correct. They do only write a 1GB image to a 1GB partition, GParted misreports the partition layout.
     
    Last edited: 30 Oct 2019
    Corky42, 30 Oct 2019
    #285
  6. Corky42

    Corky42 Where's walle?

    Joined:
    30 Oct 2012
    Posts:
    9,648
    Likes Received:
    388
    So it's taken me all day to workout what's going on hear and i just wanted to dbl check I've understood correctly, apologies if my wording isn't exact as I've got zero knowledge of programing.

    The section before the ; is setting an environment variable for the program that follows the ; in this case you associated (by using for) the letter i with any (all?) files starting with manjaro and ending in iso, then dd runs and we tell it to use the EV we just set by using the $.
     
    Corky42, 30 Oct 2019
    #286
  7. Gareth Halfacree

    Gareth Halfacree WIIGII! Lover of bit-tech Administrator Super Moderator Moderator

    Joined:
    4 Dec 2007
    Posts:
    17,130
    Likes Received:
    6,719
    It'll probably make more sense like this:

    Code:
    for i in manjaro*iso;
   do dd if=$i of=/dev/sdb status=progress;
done
    So, effectively, there are three commands. The first creates a list of filenames matching "manjaro*iso" - so anything which starts "manjaro" and ends "iso" and has something inbetween. The for loop then iterates over the list (which is why we used "i" as the variable name - to indicate it's an iterative variable.) The second then takes one of these filenames, which has been assigned the variable name i, and runs a command with said filename in it as a string ($i). When that command has finished, it'll move onto the next filename that matched, and the next, and the next, then finish (done) when it's run out of matching filenames.

    This, by contrast:

    Code:
    for i in $(ls -r manjaro*iso | head -n1);
    do dd if=$i of=/dev/sdb status=progress;
done
    Does something similar, except everything in the $() bit is a separate command whose output is treated as a string. In other words, $(echo boo) is functionally equivalent to "boo" - and $(ls -r manjaro*iso | head -n1)" is functionally equivalent to "manjaro-whateverthelastoneinthefolderis.iso".

    Because we're only using one result (head -n1), though, it's a bit of a waste of a for loop. The exact same command could be written as:

    Code:
    dd if=$(ls -r manjaro*iso | head -n1) of=/dev/sdb status=progress
    We're only working on one file, so no loop's needed.

    Alternatively...

    Code:
    dd if=$(ls manjaro*iso | tail -n1) of=/dev/sdb status=progress
    In the first version, we're asking ls to give us a list of files in reverse alphanumeric order then using head to take the first entry only; in this version, we're asking ls to give us a list of files in normal alphanumeric order then using tail to take the last entry only. Functionally, there's no difference. Told you there's always multiple ways to do things in Linux!
     
    Gareth Halfacree, 30 Oct 2019
    #287
    Corky42 likes this.
  8. Corky42

    Corky42 Where's walle?

    Joined:
    30 Oct 2012
    Posts:
    9,648
    Likes Received:
    388
    Wow, that's way more than i was expecting, thanks for taking the time to go into such detail, that's going to take some time to digest as it's taken me an entire day just trying to workout the first example you gave. :)

    Not that I'm complaining as like i said it's quiet fun trying to work these sort of things out.
     
    Corky42, 30 Oct 2019
    #288
  9. Corky42

    Corky42 Where's walle?

    Joined:
    30 Oct 2012
    Posts:
    9,648
    Likes Received:
    388
    Quick update: I tried your first example and it fell over moaning about something, something, "do" so I've ended up with the following..
    Code:
    myvar=$( ls manjaro*iso ) ; sudo dd bs=4M if=$myvar of=/dev/disk/by-id/usb-Samsung_Flash_Drive_0000000000000000-0:0 status=progress oflag=sync ; done
    I changed the id of the drive in the above into "000.." in the above because serial number. :)

    Now i just need to workout how to recreate the partition from where ^^that^^ ends to the end of the drive, i think I'll leave that for another day though.

    Thanks again for pointing me in the right direction Mr H.
     
    Corky42, 31 Oct 2019
    #289
  10. Gareth Halfacree

    Gareth Halfacree WIIGII! Lover of bit-tech Administrator Super Moderator Moderator

    Joined:
    4 Dec 2007
    Posts:
    17,130
    Likes Received:
    6,719
    Odd - seems to work for me. You using something other than bash for your shell?

    Code:
    blacklaw@shodan:/media/RAM Disk$ for i in manjaro*iso; do echo dd if=$i of=/dev/sdb status=progress; done
dd if=manjaro-test.iso of=/dev/sdb status=progress
    I'd still recommend using the $(ls manjaro*iso | tail -n1) approach, mind, 'cos as much as you'd like to think you'll remember to remove old ISOs I bet there'll come a time when you forget.

    Probably best using parted for that - 's pretty scriptable.
     
    Gareth Halfacree, 31 Oct 2019
    #290
  11. Corky42

    Corky42 Where's walle?

    Joined:
    30 Oct 2012
    Posts:
    9,648
    Likes Received:
    388
    Not afaik, 'echo $0' reports /bin/bash, if it's of interest the exact working of the error was "bash: syntax error near unexpected token `do'".
     
    Corky42, 31 Oct 2019
    #291
  12. Gareth Halfacree

    Gareth Halfacree WIIGII! Lover of bit-tech Administrator Super Moderator Moderator

    Joined:
    4 Dec 2007
    Posts:
    17,130
    Likes Received:
    6,719
    Sounds to me like there's something screwy going on. What happens if you copy and paste from this text file? (It won't do anything, there's an "echo" before "dd" so it'll just print what it *would* have done if the echo wasn't there.)
     
    Gareth Halfacree, 31 Oct 2019
    #292
  13. Corky42

    Corky42 Where's walle?

    Joined:
    30 Oct 2012
    Posts:
    9,648
    Likes Received:
    388
    It prints "dd if=manjaro-kde-18.1.1-191015-linux53.iso of=/dev/sdb status=progress" to the command line...Something is going on as if i add sudo to the start of it it errors again so I'm guessing it's something to do with that.

    Just to cast aside any did he enter sudo wrongly a copy and past from the cli...
    Code:
    [peter@peter-pc Software]$ sudo for i in manjaro*iso; do echo dd if=$i of=/dev/sdb status=progress; done
    :)
     
    Corky42, 31 Oct 2019
    #293
  14. Gareth Halfacree

    Gareth Halfacree WIIGII! Lover of bit-tech Administrator Super Moderator Moderator

    Joined:
    4 Dec 2007
    Posts:
    17,130
    Likes Received:
    6,719
    That's not where you want the sudo. Remember, it's three commands: "for," "do", and "done". "sudo for" leaves "do" running as you, not as the root user. You want "do sudo".

    Code:
    for i in manjaro*iso; do sudo echo dd if=$i of=/dev/sdb status=progress; done
    Want confirmation it's running as root?

    Code:
    for i in manjaro*iso; do sudo whoami; done
     
    Gareth Halfacree, 31 Oct 2019
    #294
  15. Corky42

    Corky42 Where's walle?

    Joined:
    30 Oct 2012
    Posts:
    9,648
    Likes Received:
    388
    D'oh!

    I suspected it was something to do with where sudo was placed so i tried it with 'for i ...; sudo do' and got the same error, if only I'd thought about it a bit more as the moment you mentioned "do sudo" i had one of those how did i miss that moments.
     
    Corky42, 31 Oct 2019
    #295
  16. Paradigm Shifter

    Paradigm Shifter de nihilo nihil fit

    Joined:
    10 May 2006
    Posts:
    2,306
    Likes Received:
    86
    To follow up a little on this; now 5.3 is the preferred kernel for Mint, temperatures are being reported on a 3900X (although I got it working by compiling an updated k10temp kernel module for 4.15 and 4.18) and AMD systems are still happy and stable.

    But I'm forced to avoid 5.3 on my Intel based boxes (if using iGPU) because otherwise I get random soft locks every so often of the display. It always recovers, but it is really annoying. Intel appear to be making their GPU drivers worse. And this was before the recent slew of mitigations.

    Impressed, I ain't.
     
    Paradigm Shifter, 31 Jan 2020
    #296
  17. Corky42

    Corky42 Where's walle?

    Joined:
    30 Oct 2012
    Posts:
    9,648
    Likes Received:
    388
    After having spent days reading stuff i could do with someone confirming my understanding of GRUB2 and UEFI, having only ever used GRUB and BIOS in the past i just want to check that I've got a rough idea of how the aforementioned does things.

    What would really help is some clarity on what GRUB2 is doing, would i be right in thinking that "grub-install" is basically creating a "bootx64.efi" on the fly using various configs and that it includes things like a kernel.img, modules to access the filesystem where the true kernel is and stuff like that?
     
    Corky42, 6 Mar 2020
    #297
  18. yuusou

    yuusou Multimodder

    Joined:
    5 Nov 2006
    Posts:
    2,878
    Likes Received:
    955
    It's more or less as you say. Grub-install on an efi system creates the efi file, which in turn starts initramfs which starts grub using the config file in /boot/grub/grub.cfg (which you usually create with grub-mkconfig) and then loads the kernel you've picked.
     
    yuusou, 6 Mar 2020
    #298
  19. Corky42

    Corky42 Where's walle?

    Joined:
    30 Oct 2012
    Posts:
    9,648
    Likes Received:
    388
    I thought the initramfs was loaded after the kernel, that the kernel created it shortly after taking over the boot process. :confused:
     
    Corky42, 6 Mar 2020
    #299
  20. yuusou

    yuusou Multimodder

    Joined:
    5 Nov 2006
    Posts:
    2,878
    Likes Received:
    955
    Maybe you're right, but there's something before grub, because I know that you can have grub and /boot encrypted.
     
    yuusou, 6 Mar 2020
    #300
Page 15 of 16
Tags:

Share This Page