LVM pvscan, vgscan & lvscan taking too long to run - Webfwd
29th December 2015
Go Back

LVM pvscan, vgscan and lvscan taking too long to run

alt

Just before Christmas, we encountered monitoring issues with two host nodes (Debian) within our virtual cluster. Although all guest virtual machines were running perfectly fine – the nodes had stopped reporting virtual machine usage to our billing platform.

On investigation, we noticed that running pvscan, vgscan or lvscan would take 60+ seconds to run which our reporting cron must run.

We tested lvm by running the command in verbose mode and noticed that it was waiting for a long time at one command.

pvscan -vv
vgscan -vv
lvscan -vv

We also noticed that /etc/lvm/archive/ contained tens of thousands of files whereas our other nodes had less than 1000 files.

After further reading, we found that this was caused by an intermittent route to one of our iSCSI SAN’s with multipath configured.

Here is how we fixed it

As we do not use LVM snapshots for restore purposes, we do not need LVM archiving.

We edited /etc/lvm/lvm.conf and changed the line

archive = 1

to

archive = 0

We restarted the lvm service by running

/etc/init.d/lvm2 restart

And then deleted all files within the archive folder with the following command find /etc/lvm/archive/ -type f -delete

If you simply try to rm -rf * within the folder then you will encounter an argument too long error as there are too many files for rm to handle.

Facebooktwittergoogle_pluspinterestlinkedinmailFacebooktwittergoogle_pluspinterestlinkedinmail
Tags: , , , , ,
About the Author

At heart, I am a geek. I started building websites at the age of 11 and have grown from a freelancer to running three successful web agencies located in Tamworth, Cardiff and Surrey, primarily through word-of-mouth. I love web design, and I like to think that it shows in our work. My specialities lie in bespoke web development, security testing and server management but I have a passionate team around me to complete the whole package and make our businesses the number one choice.

Subscribe to our mailing list and we will keep you up to date with our latest blog posts!