Setting up fresh disk for LVM

Format, partition and create file system.

System settings

Disable issuing discards for SSD drives, so LVM operations will be undoable:

CONFIG:/etc/lvm/lvm.conf

issue_discards = 0

Setting logical sector size

Warning: Changing sector size destroys all data.

List current logical and physical sector sizes for block devices:

BASH

$ lsblk -dt
NAME    ALIGNMENT MIN-IO OPT-IO PHY-SEC LOG-SEC ROTA SCHED       RQ-SIZE  RA WSAME
sda             0   4096      0    4096     512    1 mq-deadline       2 128    0B
nvme0n1         0    512      0     512     512    0 none           1023 128    0B

for SATA drives: set logical sector size equal to physical (if supported)

BASH

# hdparm -I /dev/sda | grep 'Sector size:'
Logical  Sector size:                   512 bytes [ Supported: 512 4096 ]
Physical Sector size:                  4096 bytes
# hdparm --set-sector-size 4096 --please-destroy-my-drive  /dev/sda

for NVMe drives: set logical sector size based on relative performance

BASH

# nvme id-ns -H /dev/nvme0n1 | grep 'Relative Performance'
LBA Format  0 : Metadata Size: 0   bytes - Data Size: 512 bytes - Relative Performance: 0x2 Good (in use)
LBA Format  1 : Metadata Size: 0   bytes - Data Size: 4096 bytes - Relative Performance: 0x1 Better
# nvme format --lbaf=1 /dev/nvme0n1

Further details: Advanced format – Archlinux wiki

Partitioning and LVM setup

Parition alignment on 2MiB (equal to maximum possible Erase Block Size for SSD/NVMe drives) equivalent to 4096 sectors 512B each/512 sectors 4KiB each.

BASH

# sfdisk /dev/nvme0n1
> label: gpt

Leave 2MiB of space at the beginning for parition table and (optionally) bootloader.

For system drives, reserve 1022MiB EFI partition. Firmware updates use EFI partition and may require >100MiB.

Remaining disk space is for LVM. For non-system drives, LVM partition starts at 2MiB. Values given to sfdisk are in logical sectors (as opposed to physical sectors, as they’re not necessarily same size) and example below is using 4KiB logical sector size.

BASH

> 512, 261632, U, *
> 262144, , V,
> w
> q

# pvcreate --dataalignment 2M /dev/nvme0n1p2
# pvs /dev/nvme0n1p2 -o+pe_start
# vgcreate -s 1G vgdev /dev/nvme0n1p2

(verify that pe_start is a multiple of alignment size).

If you want to extend existing VG, see here.

File system setup and mouning

Create LV, optionally add redundancy/integrity layer (details).

BASH

# lvcreate -L 2T -n backup --type raid1 --mirrors 1 vgdev
# lvconvert --raidintegrity y --raidintegrityblocksize 4096 --raidintegritymode bitmap /dev/vgdev/backup

For ext4 file systems, block sizes availavle on x86 architectures are from 4KiB to 64KiB.

Disable journaling.

For file systems on RAID devices other thatn RAID 1 (mirror), add -E stride=512,stripe_width=512, options. Stride size/stripe width is given in file system block multiplies and should be aligned same as paritions (2MiB).

-E discard can be omitted as it is the default.

BASH

# mkfs.ext4 -b 4096 -O ^has_journal /dev/vgdev/backup

Use following mount options in /etc/fstab:

(for external disks) noauto
noatime,errors=remount-ro
(for SSD only) nodiscard – discard on weekly basis using cron (make sure you’re using anacron on desktop):

BASH:/etc/cron.weekly/fstrim

#!/bin/bash
/usr/bin/ionice -c 3 /sbin/fstrim -v -a

Monitoring

LV RAID health status

Make sure that cron messages are received, e.g. sent to proper e-mail address.

BASH:/etc/cron.hourly/lv-health-status

#!/bin/sh
lvs -o full_name,lv_health_status --separator ':' | tail -n+2 | egrep -v ':[[:space:]]*$'

TODO: monitoring smartd with cron task.