We wrote about separated mountpoint Debian Install guide, and some may wonder if those amounts are proper.
We also wonder about it. So we did a little survey.
Buster status notes. Minor tweaks.
What is important
In short, the peak usage of the mountpoints are crucial.
df -h tells us how much they occupy, but it's not enough.
To achieve I/O stats per mountpoints,
user$ cat /sys/fs/cgroup/blkio/blkio.throttle.io_service_bytes
This shows Read, Write, Sync and Async per block devices.
To combine major:minor numbers and actual mountpoint names,
user$ lsblk -l
We wrote some bash scripts to manage the results in CSV and ReStructuredText tables.
The data is only Read and Write values; no Sync and Async.
- GNOME desktop machine, Debian Stretch -> Now Buster.
- Single user workstation.
- We use noatime and tmpfs to reduce write I/Os.
- Browsing and scripting, committing to local VCS repositories, and backups everyday.
We processed those results by Python + pandas.
Additional one month survey resulted in very little differences.
We "utilized XDG tmpfs more" on our scripts to reduce writes; such as backups temporary directory, script outputs, logs, etc.
ext4 data=writeback on /home does not seems significant.
A clean installed Debian Buster machine seems very significant reduction of write operations; almost expected write amounts.
We need more time and data to evaluate again.
mountpoint / /home /usr /usr/share \ count 3.100000e+01 3.100000e+01 3.100000e+01 3.100000e+01 mean 5.545155e+07 1.004920e+09 4.538749e+08 9.897872e+07 std 2.596340e+06 3.489151e+09 2.254861e+08 5.592428e+07 min 5.000909e+07 3.712000e+06 1.890847e+08 4.237824e+07 25% 5.336166e+07 1.107251e+07 2.823854e+08 5.606093e+07 50% 5.575373e+07 6.061773e+07 4.361308e+08 8.756122e+07 75% 5.736755e+07 8.877568e+07 6.311004e+08 1.357015e+08 max 6.071194e+07 1.425558e+10 8.861788e+08 2.766162e+08 mountpoint /var /var/cache /var/log /var/mail \ count 3.100000e+01 3.100000e+01 3.100000e+01 3.100000e+01 mean 3.186039e+08 2.748591e+07 4.634211e+06 3.383048e+06 std 3.716843e+07 1.180817e+07 3.528061e+06 7.430375e+04 min 2.890762e+08 6.112256e+06 2.831360e+06 3.343360e+06 25% 2.937754e+08 2.717901e+07 3.630080e+06 3.343360e+06 50% 2.958469e+08 3.344896e+07 3.867648e+06 3.351552e+06 75% 3.539016e+08 3.378893e+07 4.293632e+06 3.386368e+06 max 4.271237e+08 3.607040e+07 2.320896e+07 3.675136e+06 mountpoint /var/spool /var/tmp NVMe count 3.100000e+01 3.100000e+01 3.100000e+01 mean 3.217540e+06 4.163402e+06 1.997444e+09 std 4.397872e+05 2.089349e+05 3.648750e+09 min 2.360320e+06 3.269632e+06 6.179169e+08 25% 3.412992e+06 4.068352e+06 7.596790e+08 50% 3.421184e+06 4.191232e+06 1.010303e+09 75% 3.429376e+06 4.332544e+06 1.289692e+09 max 3.638272e+06 4.359168e+06 1.576072e+10
mountpoint / /home /usr /usr/share \ count 3.100000e+01 3.100000e+01 3.100000e+01 3.100000e+01 mean 1.112989e+06 6.388108e+08 5.060542e+05 1.055249e+06 std 1.682694e+06 1.325640e+09 2.233945e+06 4.429413e+06 min 2.867200e+04 6.635520e+05 2.867200e+04 2.867200e+04 25% 1.228800e+05 2.523341e+07 3.276800e+04 3.276800e+04 50% 5.201920e+05 2.849341e+08 3.276800e+04 3.276800e+04 75% 1.013760e+06 6.725038e+08 3.276800e+04 3.276800e+04 max 7.503872e+06 7.334855e+09 1.246413e+07 2.466816e+07 mountpoint /var /var/cache /var/log /var/mail \ count 3.100000e+01 3.100000e+01 3.100000e+01 3.100000e+01 mean 1.037924e+08 4.776801e+07 1.012425e+07 1.764385e+06 std 1.375454e+08 5.875282e+07 1.784250e+07 4.124028e+06 min 4.423680e+05 1.228800e+04 4.915200e+05 1.433600e+05 25% 1.148928e+06 3.276800e+04 1.292288e+06 2.314240e+05 50% 3.911680e+06 2.756198e+07 3.264512e+06 5.570560e+05 75% 1.711329e+08 6.883430e+07 8.062976e+06 1.046528e+06 max 5.027267e+08 1.904763e+08 7.138918e+07 1.939866e+07 mountpoint /var/spool /var/tmp NVMe count 3.100000e+01 3.100000e+01 3.100000e+01 mean 2.104089e+06 4.970496e+06 3.291994e+09 std 4.210203e+06 2.593516e+07 6.079978e+09 min 2.293760e+05 1.228800e+04 2.614272e+06 25% 3.133440e+05 2.334720e+05 8.743619e+08 50% 7.946240e+05 2.969600e+05 1.903450e+09 75% 1.552384e+06 4.546560e+05 3.662546e+09 max 1.966899e+07 1.447076e+08 3.416394e+10
Apology about hard to read raw data.
Here are some examples tables.
Just After cold boot, logged in, open a GNOME terminal.
On the other hand, below cases are just before shutdown the box.
Case 1 (Stretch)
Some AppArmor Profile tasks, mostly aa-logprof.
Case 2 (Stretch)
Tiny but many VCS commits.
Watch the amount of /home and NVMe writes. They are some Giga-Bytes operations.
- What we did was almost writing scripts and committing them on each local bzr repositories.
- Some web browsing (only-memory-cache, no disk cache.)
- No media creation.
- No software compilation.
NVMe seems to be too large at a glance. However, those mountpoints are all journaled ext4 over LVM.
Our machines are all LVM-over-LUKS, ext4 journaled.
Hence, especially VCS commits may make storage writes more than double.
VCS atomic operations, over ext4 journal operations, over LVM operations.
/var/log shows a little bit differences. This is mostly due to AppArmor audit logs.
Without them, /var/log just have 2MB/day writes on a GNOME desktop usage.
The occupation is also small, almost below 100MB under the Debian's default logrotate setting.
Considering the amounts
Then, how small the each mountpoint could be?
This section was so messy.
Please forget about pseudo-math like expressions. Very poor.
However, our conclusion is same. Use large amount SSD/NVMe for /home.
That seems the simple and effective method.
Considering peak writes and non-TRIM area, Our hypothesis is,
- Assume Mountpoint M.
- a(M) as amount of M.
- u(M) as used amount of M.
- t(M) as not discarded, TRIM target area (amount) of M.
Minimum amount of mountpoint on SSD/NVMe
a(M) = u(M) + t(M) + C (where 1GB < C.)
Let us assume u(/home) is around 4GB.
Our statistics showed /home tends to be written much than expected.
- On average, it is below 300MB, but the peak is 7GB in a day.
- If you do TRIM every day, t(/home) would be under 10GB.
- The pandas 75% line tells us /home tends to have 600MB writes per day.
So roughly, t(/home) would be,
- around 6GB per week TRIM condition.
- maybe 30GB per month TRIM condition.
Hence, a(/home) should be more than 10GB without ext4 discard option.
- If you do TRIM manually per month, it should be above 32GB.
- The ideal amount might be around 64GB or above.
Large SSD/NVMe or separated /home is better.
Buy above 500GB one, or separate /home into another storage.
The other mountpoints are not so much on desktop, but on server /var and under that should be separated with proper amounts.
Larger /home makes TRIM by fstrim takes a bit longer time.
If you do not put audio, video, pictures so much, We think 64GB is not a bad initial value.
You can just extend by LVM LV and ext4 on demand, provided you have spare area in your LVM PE.
See also: How to extend LVM LV and ext4
Workstations may not need so much amounts around /var.
Thanks for reading a long messy article.
Have a nice day.