Merge branch 'master' of https://github.com/recolic/windows2usb

Squashed commit of the following:

commit c279195a08
Author: Recolic K <bensl@microsoft.com>
Date:   Thu Nov 25 17:50:06 2021 +0800

    doc improvement: add feature matrix

commit 7b0f5d9912
Author: Recolic K <bensl@microsoft.com>
Date:   Thu Nov 25 17:43:26 2021 +0800

    doc improvement: add feature matrix

commit 60d1f7d758
Author: Recolic K <bensl@microsoft.com>
Date:   Thu Nov 25 17:41:52 2021 +0800

    doc improvement: add feature matrix

commit f8d27f725d
Author: Recolic K <bensl@microsoft.com>
Date:   Thu Nov 25 17:39:57 2021 +0800

    doc improvement: add feature matrix

Signed-off-by: ValdikSS <iam@valdikss.org.ru>

README.md

@@ -3,7 +3,7 @@
 
 ## What is this?
 
-Windows2usb is a bash script which writes Microsoft Windows 7/8/8.1/10 installation DVD images to USB Flash Drive or external HDD on Linux. It was designed with compatibility in mind, and should work in all cases, contrary to other popular Linux tools.
+Windows2usb is a bash script which writes Microsoft Windows 7/8/8.1/10/11 installation DVD images to USB Flash Drive or external HDD on Linux. It was designed with compatibility in mind, and should work in all cases, contrary to other popular Linux tools.
 
 Features:
 
@@ -13,53 +13,89 @@ Features:
 * Uses stock Windows bootloaders where possible
 * Does not break UEFI Secure Boot chain
 
+```
+Windows 7/8/8.1/10/11 ISO to Flash Drive burning utility
+WARNING: this program will delete all existing data on your drive!
+
+windows2usb <device> <windows iso> [mbr/gpt/gptntfs/gpt+uefintfs]
+
+mbr mode: the most universal, RECOMMENDED and DEFAULT method.
+   This mode creates MBR partition table with FAT32 partition,
+   installs BIOS and UEFI bootloaders, supports Secure Boot.
+   install.wim file larger than 4 GiB will be split.
+   Suitable for all computers (UEFI/CSM/BIOS).
+
+gpt mode: less universal mode, for modern (UEFI) computers.
+   GPT+FAT32, UEFI only, supports Secure Boot.
+
+gptntfs mode: all the same as 'gpt' but NTFS is used.
+   GPT+NTFS, UEFI only, supports Secure Boot.
+   Large install.wim file will not be split.
+   NOTE: not all UEFI are compatible with this mode,
+   NTFS driver should be present on the motherboard.
+
+gpt+uefintfs mode: alternative hacky installation method, not recommended.
+   This mode uses NTFS partition and third-party 'uefintfs' bootloader.
+   GPT+NTFS(data)+FAT32(efi), UEFI only, NO Secure Boot support.
+   Large install.wim file will not be split.
+```
+
 ## How to use?
 
 Download latest [portable AppImage version](https://github.com/ValdikSS/windows2usb/releases) from the **Releases** page, set *execution bit* (`chmod +x windows2usb-*.AppImage`) and run it from the terminal.
 
-`windows2usb <device> <windows iso> <mbr/gpt/gptntfs>`
+`windows2usb <device> <windows iso>`
 
-The program prints removable storage list if no arguments supplied.
+For example:
+
+`./windows2usb.AppImage /dev/sdz /home/valdikss/windows10.iso`
+
+The program prints removable storage list if no arguments are supplied.
+
+If you don't want to use AppImage, you'll need to install all dependencies and download [uefi-ntfs.img](https://github.com/pbatard/rufus/tree/master/res/uefi) from Rufus project.
+
+### Feature Matrix
+
+|Modes       |Legacy/UEFI-CSM Boot|UEFI Boot|Large ISO (>4GB)|Secure Boot|File System|Part Table|
+|:----------:|:------------------:|:-------:|:--------------:|:---------:|:---------:|:--------:|
+|mbr(hybrid) |Supported           |Supported|Supported       |Supported  |FAT32      |MBR       |
+|gpt         |No                  |Supported|Supported       |Supported  |FAT32      |GPT       |
+|gptntfs     |No                  |Partial  |Supported       |Supported  |NTFS       |GPT       |
+|gpt+uefintfs|No                  |Supported|Supported       |No         |NTFS       |GPT       |
 
 ### BIOS Boot
 
-BIOS Boot (Legacy Boot/UEFI-CSM) uses stock Windows 7 MBR and NTFS bootloader, courtesy of [ms-sys](http://ms-sys.sourceforge.net/) project.
+BIOS Boot (Legacy Boot/UEFI-CSM) uses stock Windows 7 MBR and FAT32 bootloader, courtesy of [ms-sys](http://ms-sys.sourceforge.net/) project.
 
-Use this mode if you have old computer without UEFI support or want maximum compatibility of installation media.
+**BIOS boot is supported only in recommended 'mbr' mode.**
 
-To burn ISO in this mode, run:
+To burn ISO compatible with BIOS boot (and UEFI boot as well), run:
 
 `windows2usb <device> <windows iso> mbr`
 
 ### UEFI Boot
 
-UEFI Boot (`gpt` mode) creates FAT32 partition with stock Windows UEFI bootloader.
-
-This mode will not work on old computers. Use this mode for new computers with UEFI support.
-
-This mode supports UEFI Secure Boot.
+**UEFI Boot is supported in all modes. Recommended mode: 'mbr'.**
 
 To burn ISO in this mode, run:
 
-`windows2usb <device> <windows iso> gpt`
+`windows2usb <device> <windows iso> mbr`
 
-### UEFI Boot with NTFS Partition
+'MBR', 'GPT' and 'GPTNTFS' modes utilize Windows UEFI bootloader from the ISO file, with Secure Boot support.
 
-UEFI Boot with NTFS partition uses [uefi-ntfs](https://github.com/pbatard/uefi-ntfs) bootloader from Rufus project.
+### UEFI Boot with NTFS Partition and uefi-ntfs bootloader
 
-This mode is made for custom installation disks with install.wim file greater than 4 GiB, which could be found on various torrent trackers. **Windows2usb** creates 2 partitions in this mode, small 1 MiB FAT32 partition with **uefi-ntfs** and huge NTFS partition with ISO data.
+A special 'gpt+uefintfs' mode uses [uefi-ntfs](https://github.com/pbatard/uefi-ntfs) bootloader from Rufus project. **Windows2usb** creates 2 partitions in this mode, small 1 MiB FAT32 partition with **uefi-ntfs** and huge NTFS partition with ISO data.
+
+This mode was included earlier to work with installation disks where install.wim file is greater than 4 GiB, but since then split WIM file functionality was introduced, and now this mode may be considered obsolete.
 
 This mode does not support Secure Boot (uefi-ntfs bootloader is not signed by Microsoft or other trusted party).
 
-To burn ISO in this mode, run:
-
-`windows2usb <device> <windows iso> gptntfs`
-
-If your ISO contains install.wim greater than 4 GiB, `gpt` mode will automatically detect that and terminate the writing process.
-
 ## Alternatives
 
-[WoeUSB](https://github.com/slacka/WoeUSB)—does not support NTFS partitioning and install.wim > 4GiB; uses GRUB for BIOS Boot.
+[WoeUSB](https://github.com/WoeUSB/WoeUSB)—uses GRUB for BIOS Boot.  
+[bootiso](https://github.com/jsamr/bootiso/)—supports Windows and Linux ISOs.  
+[Ventoy](https://www.ventoy.net/en/index.html)—Booting multi-tool with brilliant support of Windows ISO direct loading.
 
 ## Credits
 
@@ -68,3 +104,5 @@ This script uses:
 * **lsblk** and **sfdisk** from [util-linux](https://mirrors.edge.kernel.org/pub/linux/utils/util-linux/)
 * [**ms-sys**](http://ms-sys.sourceforge.net/) for native Windows 7 MBR and NTFS bootloaders
 * [**p7zip**](https://www.7-zip.org/) for ISO extraction
+* [**autofsync**](https://github.com/i-rinat/autofsync/) to prevent filesystem bufferbloat and properly show copying progress (AppImage only)
+* [**wimlib**](https://wimlib.net/) to split large install.wim files to fit FAT32 partition

windows2usb

@@ -1,10 +1,10 @@
-#!/bin/bash
+#!/usr/bin/env bash
 set -e # errexit
 set -f # noglob
 set +o braceexpand # disable braceexpand
 shopt -s nocasematch # for "if labeltype"
 
-export LANG=C
+export LANG=C.utf8
 
 bold="$(tput bold)"
 normal="$(tput sgr0)"
@@ -13,7 +13,7 @@ dirpath="$(dirname "$scriptpath")"
 
 
 function check_requirements() {
-    local reqs=(awk lsblk 7z mkfs.vfat mkfs.ntfs sfdisk ms-sys mktemp)
+    local reqs=(awk lsblk 7z mkfs.vfat mkfs.ntfs sfdisk ms-sys mktemp wimsplit)
 
     for req in ${reqs[*]}; do
         if ! command -v "$req" > /dev/null;
@@ -24,31 +24,56 @@ function check_requirements() {
     done
 }
 
+function check_iso_and_device() {
+    if [ ! -f "$1" ]
+    then
+        echo "${bold} == ERROR: ISO file not found! ==${normal}"
+        exit 106
+    fi
+
+    if [ ! -b "$2" ]
+    then
+        echo "${bold} == ERROR: Device file not found! ==${normal}"
+        exit 107
+    fi
+}
+
 function list_removable_drives() {
-    lsblk -d -I 8 -o RM,NAME,SIZE,MODEL | \
-        awk '($1 == 1) {print "/dev/" substr($0, index($0, $2))}'
+    lsblk=$(lsblk -d -I 8 -o RM,NAME,SIZE,MODEL | \
+        awk '($1 == 1) {print "/dev/" substr($0, index($0, $2))}')
+    if [[ "$lsblk" ]]; then
+        echo "$lsblk"
+    else
+        echo "No removable storage found!"
+    fi
 }
 
 function format_drive() {
     if [[ "$1" == "dos" ]] || [[ "$1" == "gpt" ]];
     then
         echo "label: $1" | sfdisk "$2"
+        sleep 3
     else
         echo "${bold} == format_drive: INTERNAL ERROR ==${normal}"
         exit 101
     fi
 }
 
+function get_iso_name() {
+    7z l "$1" | awk '/Comment = / {print $3; exit 0}'
+}
+
 function check_installwim_gt4gib() {
-    7z l "$1" | awk '($6 ~ /install.wim/) {if ($4 > 429496296) {exit 1}; exit}'
+    7z l "$1" | awk 'BEGIN {doend=1} ($6 ~ /install.wim/) {if ($4 > 4294967296) {doend=0; exit 0}; doend=0; exit 1} END {if (doend) {exit 1}}'
 }
 
 function create_partitions() {
     local mbrscript="- - 7 *"
-    local gptscript="- - U *"
-    local gptntfsscript="- 1MiB U *\n- - EBD0A0A2-B9E5-4433-87C0-68B6B72699C7 *"
+    local gptscript="- - EBD0A0A2-B9E5-4433-87C0-68B6B72699C7 *"
+    local gptuefintfsscript="- 1MiB U *\n- - EBD0A0A2-B9E5-4433-87C0-68B6B72699C7 *"
 
-    if [[ "$1" == "dos" ]] || [[ "$1" == "gpt" ]] || [[ "$1" == "gptntfs" ]];
+    if [[ "$1" == "dos" ]] || [[ "$1" == "gpt" ]] || [[ "$1" == "gptntfs" ]] \
+       || [[ "$1" == "gpt+uefintfs" ]];
     then
         if [[ "$1" == "dos" ]];
         then
@@ -60,43 +85,126 @@ function create_partitions() {
 
         elif [[ "$1" == "gptntfs" ]];
         then
-            echo -e "$gptntfsscript" | sfdisk "$2"
+            echo -e "$gptscript" | sfdisk "$2"
+
+        elif [[ "$1" == "gpt+uefintfs" ]];
+        then
+            echo -e "$gptuefintfsscript" | sfdisk "$2"
         fi
     else
         echo "${bold} == create_partitions: INTERNAL ERROR ==${normal}"
         exit 102
     fi
+    # Wait 1 second to settle new partition table
+    # See bug #24
+    sleep 1
+}
+
+function get_dev_partition_num() {
+    # $1 - device
+    # $2 - partition number
+
+    if [ -b "${1}${2}" ];
+    then
+        echo "${1}${2}"
+        return
+    fi
+
+    if [ -b "${1}p${2}" ];
+    then
+        echo "${1}p${2}"
+        return
+    fi
+
+    echo "${bold} == get_dev_partition_num: INTERNAL ERROR ==${normal}"
+    exit 105
 }
 
 function write_uefintfs() {
-    cat "$dirpath/uefi-ntfs.img" > "$1"
+    if [ -a "$dirpath/uefi-ntfs.img" ]; then
+        cat "$dirpath/uefi-ntfs.img" > "$1"
+    elif [ -a "/usr/share/windows2usb/uefi-ntfs.img" ]; then
+        cat "/usr/share/windows2usb/uefi-ntfs.img" > "$1"
+    fi
 }
 
 function extract_iso() {
     # $1 - isopath
     # $2 - destdir
+    # $3 - exclude install.wim extraction flag
 
-    7z x "$1" -o"$2"
+    if [[ "$3" == "skipinstallwim" ]]; then
+        7z x "$1" -o"$2" '-x!sources/install.wim'
+    else
+        7z x "$1" -o"$2"
+    fi
+}
+
+function split_wim() {
+    # $1 - isopath
+    # $2 - isomountpath
+    # $3 - destdir
+
+    mount -o ro "$1" "$2"
+    wimsplit "$2/sources/install.wim" "$3/sources/install.swm" 3800
+    umount "$2"
+}
+
+function extract_bootmgfw_from_installwim() {
+    # $1 - isopath
+    # $2 - isomountpath
+    # $3 - destdir
+
+    local fpath="$2/sources/install.wim"
+    [ ! -a "$fpath" ] && fpath="$2/sources/install.esd"
+
+    mount -o ro "$1" "$2"
+    7z e "$fpath" -o"$3/efi/boot/" '?/Windows/Boot/EFI/bootmgfw.efi'
+    umount "$2"
 }
 
 function umount_rm_path() {
-    umount "$1" || true
-    rm -r "$1"
+    if [ -d "$1" ]; then
+        umount "$1" || true
+        rm -r "$1"
+    fi
+
+    if [ -d "$2" ]; then
+        umount "$2" 2>/dev/null || true
+        rm -r "$2"
+    fi
 }
 
 function sigint_handler() {
-    umount_rm_path "$partpath"
+    umount_rm_path "$partpath" "$isomountpath"
 }
 
-if [[ ! "$3" ]];
+if [[ ! "$2" ]];
 then
-    echo "Windows 7/8/8.1/10 ISO to Flash Drive burning utility"
-    echo "$(basename "$0")" "<device> <windows iso> <mbr/gpt/gptntfs>"
+    echo "${bold}Windows 7/8/8.1/10/11 ISO to Flash Drive burning utility"
+    echo "WARNING: this program will delete all existing data on your drive!${normal}"
     echo
-    echo "Use MBR mode for old computers with BIOS (without UEFI), or for UEFI legacy mode (CSM)."
-    echo "Use GPT mode for new UEFI computers."
-    echo "Use 'gptntfs' mode for custom ISO images with install.wim file greater than 4 GiB."
-    echo "NOTE: gptntfs does not support Secure Boot!"
+    echo "$(basename "$0")" "<device> <windows iso> [mbr/gpt/gptntfs/gpt+uefintfs]"
+    echo
+    echo "mbr mode: the most universal, RECOMMENDED and DEFAULT method."
+    echo "   This mode creates MBR partition table with FAT32 partition,"
+    echo "   installs BIOS and UEFI bootloaders, supports Secure Boot."
+    echo "   install.wim file larger than 4 GiB will be split."
+    echo "   Suitable for all computers (UEFI/CSM/BIOS)."
+    echo
+    echo "gpt mode: less universal mode, for modern (UEFI) computers."
+    echo "   GPT+FAT32, UEFI only, supports Secure Boot."
+    echo
+    echo "gptntfs mode: all the same as 'gpt' but NTFS is used."
+    echo "   GPT+NTFS, UEFI only, supports Secure Boot."
+    echo "   Large install.wim file will not be split."
+    echo "   NOTE: not all UEFI are compatible with this mode,"
+    echo "   NTFS driver should be present on the motherboard."
+    echo
+    echo "gpt+uefintfs mode: alternative hacky installation method, not recommended."
+    echo "   This mode uses NTFS partition and third-party 'uefintfs' bootloader."
+    echo "   GPT+NTFS(data)+FAT32(efi), UEFI only, NO Secure Boot support."
+    echo "   Large install.wim file will not be split."
     echo
     echo "${bold} == Removable storage list ==${normal}"
 
@@ -108,81 +216,139 @@ if [[ "$EUID" == "0" ]];
 then
     dev="$1"
     isopath="$2"
+    isolabel=""
     labeltype="$3"
-    partpath="$(mktemp -d /run/winiso-write.XXXXXXXXXX)"
+    [[ ! "$labeltype" ]] && labeltype="mbr"
+    mountntfs_cmd="mount.ntfs-3g"
+    if ! command -v "$mountntfs_cmd" > /dev/null;
+    then
+        mountntfs_cmd="mount"
+    fi
 
     check_requirements
-    trap sigint_handler INT
+    check_iso_and_device "$isopath" "$dev"
 
-    # MBR
+    isolabel="$(get_iso_name "$isopath")"
+    if [ $? -ne 0 ]
+    then
+        isolabel=""
+    fi
+    echo "${bold} == Working with ISO $isolabel ==${normal}"
+
+    skipinstallwim=""
+    splitinstallwim=1
+    partpath="$(mktemp -d /run/windows2usb.XXXXXXXXXX)"
+    isomountpath="$(mktemp -d /run/windows2usb-mount.XXXXXXXXXX)"
+    trap sigint_handler INT EXIT
+
+    # MBR FAT32
     if [[ "$labeltype" == "mbr" ]];
     then
         echo "${bold} == Creating new MBR-formatted partition table ==${normal}"
         format_drive "dos" "$dev"
 
-        echo "${bold} == Creating NTFS partition ==${normal}"
+        echo "${bold} == Creating FAT partition ==${normal}"
         create_partitions "dos" "$dev"
-        mkfs.ntfs -f "${dev}1"
+        mkfs.vfat -n "${isolabel:0:11}" "$(get_dev_partition_num "${dev}" "1")"
 
         echo "${bold} == Writing bootloader ==${normal}"
-        ms-sys -7 "${dev}"
-        ms-sys -n "${dev}1"
+        ms-sys -7 -f "${dev}"
+        ms-sys -e "$(get_dev_partition_num "${dev}" "1")"
 
         echo "${bold} == Mounting data partition ==${normal}"
-        mount "${dev}1" "$partpath"
+        mount -o utf8=true "$(get_dev_partition_num "${dev}" "1")" "$partpath"
 
-    # GPT
+    # GPT FAT32
     elif [[ "$labeltype" == "gpt" ]];
     then
-        if check_installwim_gt4gib "$isopath";
-        then
-            echo "${bold} == ERROR: install.wim is greater than 4 GiB ==${normal}"
-            echo "${bold} == ERROR: Please use gptntfs mode ==${normal}"
-            umount_rm_path "$partpath"
-            exit 103
-        fi
         echo "${bold} == Creating new GPT-formatted partition table ==${normal}"
         format_drive "gpt" "$dev"
 
-        echo "${bold} == Creating UEFI FAT partition ==${normal}"
+        echo "${bold} == Creating FAT partition ==${normal}"
         create_partitions "gpt" "$dev"
-        mkfs.vfat "${dev}1"
+        mkfs.vfat -n "${isolabel:0:11}" "$(get_dev_partition_num "${dev}" "1")"
 
         echo "${bold} == Mounting data partition ==${normal}"
-        mount "${dev}1" "$partpath"
+        mount -o utf8=true "$(get_dev_partition_num "${dev}" "1")" "$partpath"
 
-    # GPT FAT32 + NTFS
+    # GPT NTFS
     elif [[ "$labeltype" == "gptntfs" ]];
     then
+        splitinstallwim="" # do NOT split install.wim in this mode
+
+        echo "${bold} == Creating new GPT-formatted partition table ==${normal}"
+        format_drive "gpt" "$dev"
+
+        echo "${bold} == Creating Microsoft NTFS partition ==${normal}"
+        create_partitions "gptntfs" "$dev"
+        mkfs.ntfs -L "$isolabel" -f "$(get_dev_partition_num "${dev}" "1")"
+
+        echo "${bold} == Mounting data partition ==${normal}"
+        "$mountntfs_cmd" "$(get_dev_partition_num "${dev}" "1")" "$partpath"
+
+    # GPT FAT32 (UEFINTFS) + NTFS
+    elif [[ "$labeltype" == "gpt+uefintfs" ]];
+    then
+        splitinstallwim="" # do NOT split install.wim in this mode
+
         echo "${bold} == Creating new GPT-formatted partition table ==${normal}"
         format_drive "gpt" "$dev"
 
         echo "${bold} == Creating UEFI FAT and Microsoft NTFS partitions ==${normal}"
-        create_partitions "gptntfs" "$dev"
-        write_uefintfs "${dev}1"
-        mkfs.ntfs -f "${dev}2"
+        create_partitions "gpt+uefintfs" "$dev"
+        write_uefintfs "$(get_dev_partition_num "${dev}" "1")"
+        mkfs.ntfs -L "$isolabel" -f "$(get_dev_partition_num "${dev}" "2")"
 
         echo "${bold} == Mounting data partition ==${normal}"
-        mount "${dev}2" "$partpath"
+        "$mountntfs_cmd" "$(get_dev_partition_num "${dev}" "2")" "$partpath"
+    fi
+
+    if [[ "$splitinstallwim" ]] && check_installwim_gt4gib "$isopath";
+    then
+        echo "${bold} == NOTE: install.wim is greater than 4 GiB and will be split to fit FAT32 limit ==${normal}"
+        skipinstallwim="skipinstallwim"
     fi
 
     echo "${bold} == Extracting files from ISO to the partition ==${normal}"
-    extract_iso "$isopath" "$partpath"
+    extract_iso "$isopath" "$partpath" "$skipinstallwim"
+
+    if [[ "$skipinstallwim" ]]; then
+        split_wim "$isopath" "$isomountpath" "$partpath"
+    fi
+
+    # If there's no .efi bootloader files inside the iso,
+    # extract them from install.wim
+    # This is true for older Windows 7 ISO files.
+    if [ ! -f "$partpath/efi/boot/bootx64.efi" ] && \
+       [ ! -f "$partpath/efi/boot/bootia32.efi" ];
+    then
+        echo "${bold} == Extracting UEFI bootloader from install.wim ==${normal}"
+        mkdir -p "$partpath/efi/boot/" || true
+        extract_bootmgfw_from_installwim "$isopath" "$isomountpath" "$partpath"
+        mv "$partpath/efi/boot/bootmgfw.efi" "$partpath/efi/boot/bootx64.efi"
+        cp "$partpath/efi/boot/bootx64.efi" "$partpath/efi/boot/bootia32.efi"
+    fi
 
     echo "${bold} == Unmounting partition ==${normal}"
-    umount_rm_path "$partpath"
+    echo "NOTE: If this process takes very long to complete, your system is misconfigured!"
+    echo "More info: https://github.com/ValdikSS/windows2usb/issues/3#issuecomment-771534058"
+    umount_rm_path "$partpath" "$isomountpath"
 
     echo "${bold} == All done! ==${normal}"
 else
     if [[ "$APPIMAGE" ]];
     then
         scriptpath="$APPIMAGE"
-    fi;
+    fi
+
+    [ -f "$1" ] && PARAM1="$(readlink -f -- "$1")" || PARAM1="$1"
+    [ -f "$2" ] && PARAM2="$(readlink -f -- "$2")" || PARAM2="$2"
+    shift; shift;
 
     privescs=(pkexec sudo)
     for privesc in ${privescs[*]}; do
         if command -v "$privesc" > /dev/null; then
-            "$privesc" "$scriptpath" "$@"
+            "$privesc" "$scriptpath" "$PARAM1" "$PARAM2" "$@"
             exit
         fi
     done