Software-Raid https://tech.feedyourhead.at/ en Btrfs: Turning a mdadm-array into a btrfs https://tech.feedyourhead.at/content/btrfs-turning-mdadm-array-btrfs <span class="field field--name-title field--type-string field--label-hidden">Btrfs: Turning a mdadm-array into a btrfs</span> <div class="clearfix text-formatted field field--name-body field--type-text-with-summary field--label-hidden field__item"><p><img alt="" src="http://tech.feedyourhead.at/sites/tech.feedyourhead.at/files/pictures/Btrfs_logo.png" /></p> <p>Btrfs is supposed to be the "better filesystem". It is easy to use and has a nice feature-list. Since I had some troubles with a raid1 using mdadm in the past, I will try to turn my private raid-array in a btrfs. In this article I will not post the mdadm-commands(and lvm2 commands)...</p> <p>First I just remove one disk from my array and format it with btrfs:</p> <pre> <code> mkfs.btrfs /dev/sdc </code></pre> <p>Now I have a single disk in my btrfs. The cool thing about btrfs is, that I can convert my btrfs to any supported raid during runtime. This feature I want to test. So first I will mount mount btrfs but I want to use compression on my filesystem. It's super easy to use compression since it's just a mount-option. I just created an entry in my /etc/fstab:</p> <pre> <code> /dev/sdc /mnt/Raid btrfs compress=zlib,defaults 0 0 </code></pre> <p>Now I can mount my btrfs:</p> <pre> <code> mount /mnt/Raid </code></pre> <p>Now I can easily copy(for example using rsync) my data from the old mdadm-raid to my btrfs. Once this is done, I will just delete my mdadm-array and destroy the raid-meta-data using --zero-superblock(<strong>I made a backup of all my data before!</strong>). After that I have a plain disk(/dev/sdb) and I will add this disk to my btrfs:</p> <pre> <code> btrfs device add /dev/sdb /mnt/Raid/ </code></pre> <p>Finally I was very courious if I can convert the btrfs into a raid1 using the following command:</p> <pre> <code> btrfs balance start -dconvert=raid1 -mconvert=raid1 /mnt/Raid </code></pre> <p>This job runs in foreground so I would recommend using screen for it. It might take a long time. If you want to monitor the progress you can just use this command:</p> <pre> <code> watch -n1 btrfs fi df /mnt/Raid </code></pre> <p>The following one-shot-command gives a good overview about your btrfs:</p> <pre> <code> btrfs fi show; btfs fi df /mnt/Raid </code></pre> <p>It really worked like expected. I just converted my single btrfs to a raid1 using 2 disks. But I have one problem: the filesystem is mounted using this fstab-entry:</p> <pre> <code> /dev/sdc /mnt/Raid btrfs compress=zlib,defaults 0 0 </code></pre> <p>What if /dev/sdc gets broken? Then I would have to manually mount /dev/sdb. It does not matter which device I do mount since both get synchronized but if the device doesn't exist, the system would not be able to mount my filesystem. So I created a filesystem-label:</p> <pre> <code> btrfs filesystem label /mnt/Raid/ RAIDBUTTER </code></pre> <p>And I changed my fstab-entry into:</p> <pre> <code> LABEL=RAIDBUTTER /mnt/Raid btrfs compress=zlib,defaults 0 0 </code></pre> <p>It's not the end for me. Until now, I mounted my mdadm-raid and on this partition I had a directory for all my home-directories. This directory I mounted using bind-mount into /home. It was good enough for me, but it just feels like a dirty hack. Btrfs gives me the opportunity to create subvolumes. Those subvolumes I can mount wherever I want and it is just a directory in my normal btrfs. I created my subvolume using this command:</p> <pre> <code> btrfs subvolume create /mnt/Raid/Home </code></pre> <p>At last I can mount it using this fstab-line:</p> <pre> <code> LABEL=RAIDBUTTER /home btrfs subvol=Home,compress=zlib,auto,user 0 0 </code></pre> <p>Great, isn't it? In summary I can say that I really like the btrfs. It is very easy to use and has a nice feature list.</p> </div> <span class="field field--name-uid field--type-entity-reference field--label-hidden"><span lang="" about="/users/hoti" typeof="schema:Person" property="schema:name" datatype="">Hoti</span></span> <span class="field field--name-created field--type-created field--label-hidden">Aug 30 2015</span> <div class="field field--name-field-tagies field--type-entity-reference field--label-above"> <div class="field__label">Tags</div> <div class='field__items'> <div class="field__item"><i class="fa fa-tags"></i> <a href="/tags/filesystem" hreflang="en">Filesystem</a></div> <div class="field__item"><i class="fa fa-tags"></i> <a href="/Linux" hreflang="en">Linux</a></div> <div class="field__item"><i class="fa fa-tags"></i> <a href="/tags/sysadmin" hreflang="en">Sysadmin</a></div> <div class="field__item"><i class="fa fa-tags"></i> <a href="/tags/btrfs" hreflang="en">Btrfs</a></div> <div class="field__item"><i class="fa fa-tags"></i> <a href="/tags/software-raid" hreflang="en">Software-Raid</a></div> </div> </div> <section class="field field--name-comment-node-blog field--type-comment field--label-hidden comment-wrapper"> </section> <section class="field field--name-comment field--type-comment field--label-above comment-wrapper"> <h2 class='title comment-form__title'> <i class="fa fa-comments-o"></i> Add new comment</h2> <drupal-render-placeholder callback="comment.lazy_builders:renderForm" arguments="0=node&amp;1=102&amp;2=comment&amp;3=comment" token="F9wTP5enL8AmVIRvOmgO7TVSiZzXEZgCZECe7Mia3jg"></drupal-render-placeholder> </section> Sun, 30 Aug 2015 18:58:12 +0000 Hoti 102 at https://tech.feedyourhead.at mdadm: resync=pending https://tech.feedyourhead.at/content/mdadm-resyncpending <span class="field field--name-title field--type-string field--label-hidden">mdadm: resync=pending</span> <div class="clearfix text-formatted field field--name-body field--type-text-with-summary field--label-hidden field__item"><p>Today I discoverd a problem with my Software-Raid. The Raid-Array stopped rebuilding with resync=PENDING.</p> <p>&nbsp;</p> <p>cat /proc/mdadm:</p> <pre> <code> md0 : active (auto-read-only) raid1 sda1[0] sdb1[1] 8387572 blocks super 1.2 [2/2] [UU] resync=PENDING </code></pre> <p>I have no idea how this raid became like this, but i solved it with the following command:</p> <pre> <code> mdadm --readwrite /dev/md0 </code></pre> <p>After that it started rebuilding:</p> <pre> <code> md0 : active raid1 sda1[0] sdb1[1] 8387572 blocks super 1.2 [2/2] [UU] [===================&gt;.] resync = 98.3% (8245632/8387572) finish=0.0min speed=76454K/sec </code></pre> </div> <span class="field field--name-uid field--type-entity-reference field--label-hidden"><span lang="" about="/users/hoti" typeof="schema:Person" property="schema:name" datatype="">Hoti</span></span> <span class="field field--name-created field--type-created field--label-hidden">Apr 08 2015</span> <div class="field field--name-field-tagies field--type-entity-reference field--label-above"> <div class="field__label">Tags</div> <div class='field__items'> <div class="field__item"><i class="fa fa-tags"></i> <a href="/tags/software-raid" hreflang="en">Software-Raid</a></div> <div class="field__item"><i class="fa fa-tags"></i> <a href="/Linux" hreflang="en">Linux</a></div> <div class="field__item"><i class="fa fa-tags"></i> <a href="/tags/sysadmin" hreflang="en">Sysadmin</a></div> </div> </div> <section class="field field--name-comment-node-blog field--type-comment field--label-hidden comment-wrapper"> </section> <section class="field field--name-comment field--type-comment field--label-above comment-wrapper"> <h2 class='title comment-form__title'> <i class="fa fa-comments-o"></i> Add new comment</h2> <drupal-render-placeholder callback="comment.lazy_builders:renderForm" arguments="0=node&amp;1=73&amp;2=comment&amp;3=comment" token="7FycrUQswYBd0O_Ooie4CmmRDBiCYg9EsoCa7NQwC44"></drupal-render-placeholder> </section> Wed, 08 Apr 2015 12:26:34 +0000 Hoti 73 at https://tech.feedyourhead.at mdadm: throttling the resync activity https://tech.feedyourhead.at/content/mdadm-throttling-resync-activity <span class="field field--name-title field--type-string field--label-hidden">mdadm: throttling the resync activity</span> <div class="clearfix text-formatted field field--name-body field--type-text-with-summary field--label-hidden field__item"><p>If a raid is rebuilding disk access is mostly very slow. Sometimes it's better to slow down the rebuild-process for a while. This can easily done by:</p> <pre> <code> echo 100 &gt; /proc/sys/dev/raid/speed_limit_max </code></pre> Default value for speed_limit_max is on my system 200000</div> <span class="field field--name-uid field--type-entity-reference field--label-hidden"><span lang="" about="/users/hoti" typeof="schema:Person" property="schema:name" datatype="">Hoti</span></span> <span class="field field--name-created field--type-created field--label-hidden">Mar 25 2015</span> <div class="field field--name-field-tagies field--type-entity-reference field--label-above"> <div class="field__label">Tags</div> <div class='field__items'> <div class="field__item"><i class="fa fa-tags"></i> <a href="/tags/software-raid" hreflang="en">Software-Raid</a></div> <div class="field__item"><i class="fa fa-tags"></i> <a href="/Linux" hreflang="en">Linux</a></div> <div class="field__item"><i class="fa fa-tags"></i> <a href="/tags/sysadmin" hreflang="en">Sysadmin</a></div> </div> </div> <section class="field field--name-comment-node-blog field--type-comment field--label-hidden comment-wrapper"> </section> <section class="field field--name-comment field--type-comment field--label-above comment-wrapper"> <h2 class='title comment-form__title'> <i class="fa fa-comments-o"></i> Add new comment</h2> <drupal-render-placeholder callback="comment.lazy_builders:renderForm" arguments="0=node&amp;1=69&amp;2=comment&amp;3=comment" token="IQxni0p78o9ZSEWkihFEC7fFRbIcXcDos4BQLuUqJuM"></drupal-render-placeholder> </section> Wed, 25 Mar 2015 14:40:35 +0000 Hoti 69 at https://tech.feedyourhead.at Software-Raid Fork https://tech.feedyourhead.at/content/software-raid-fork <span class="field field--name-title field--type-string field--label-hidden">Software-Raid Fork</span> <div class="clearfix text-formatted field field--name-body field--type-text-with-summary field--label-hidden field__item"><p>Anything bad happend to my software RAID(1). A disk was okay but I got the following errors:</p> <pre> <code> Mar 18 20:49:12 hetz kernel: [466421.061376] sd 2:0:0:0: [sda] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK Mar 18 20:49:12 hetz kernel: [466421.061410] sd 2:0:0:0: [sda] CDB: Write(10): 2a 00 84 10 10 08 00 00 02 00 Mar 18 20:49:12 hetz kernel: [466421.061556] sd 2:0:0:0: [sda] Unhandled error code Mar 18 20:49:12 hetz kernel: [466421.061574] sd 2:0:0:0: [sda] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK Mar 18 20:49:12 hetz kernel: [466421.061607] sd 2:0:0:0: [sda] CDB: Write(10): 2a 00 04 10 10 08 00 00 02 00 </code></pre> <p>I contacted my hosting-provider. They checked the disk and told me that it OKAY. Until then there wasn't a problem. But because of this error, the disk jumped out of it's raid-array and after I rebooted the server I had all Raid-Arrays twice!! And the really big problem was, that I also have an LVM Volume-Group.</p> <p>pvdisplay:</p> <pre> <code> Found duplicate PV O0Ju3vera1nHsBj8sXxLauJkwc5GCIoR: using /dev/md127 not /dev/md3 </code></pre> <p>This is aweful. /dev/md3 should be in use instead of /dev/md127. /dev/md127 is the new forked raid-array. So I created a table to be sure which partition is in use and which raid is out of sync:</p> <pre> <code> /dev/md0 -&gt; swap /dev/md1 -&gt; /boot /dev/md2 -&gt; / /dev/md127 -&gt; volg1 /dev/md3 -&gt; OUT OF SYNC /dev/md124 -&gt; OUT OF SYNC /dev/md125 -&gt; OUT OF SYNC /dev/md126 -&gt; OUT OF SYNC </code></pre> <p>I had to do the following:</p> <ol> <li>mdadm --manage --stop /dev/MD-RAID</li> <li>mdadm --zero-superblock /dev/DISK</li> <li>mdadm --add /dev/EXISTING-MD-RAID /dev/DISK</li> </ol> <p>This isn't complicated stuff, but it failed just on the first MD-RAID. Every time I tried to stop the raidarray, I received the error-message: "md-raid in use". But mount and lsof showed me the opposit. After searching and shouting for a while, I found the solution: this raid was used as SWAP</p> <p>So I just stopped using it as swap:</p> <pre> <code> swapoff /dev/md3 </code></pre> <p>Now I was able to remove all OUT-OF-SYNC-Arrays, and add them to my working RAID. What a challange..</p> </div> <span class="field field--name-uid field--type-entity-reference field--label-hidden"><span lang="" about="/users/hoti" typeof="schema:Person" property="schema:name" datatype="">Hoti</span></span> <span class="field field--name-created field--type-created field--label-hidden">Mar 23 2015</span> <div class="field field--name-field-tagies field--type-entity-reference field--label-above"> <div class="field__label">Tags</div> <div class='field__items'> <div class="field__item"><i class="fa fa-tags"></i> <a href="/Linux" hreflang="en">Linux</a></div> <div class="field__item"><i class="fa fa-tags"></i> <a href="/tags/sysadmin" hreflang="en">Sysadmin</a></div> <div class="field__item"><i class="fa fa-tags"></i> <a href="/tags/software-raid" hreflang="en">Software-Raid</a></div> <div class="field__item"><i class="fa fa-tags"></i> <a href="/tags/lvm" hreflang="en">LVM</a></div> </div> </div> <section class="field field--name-comment-node-blog field--type-comment field--label-hidden comment-wrapper"> </section> <section class="field field--name-comment field--type-comment field--label-above comment-wrapper"> <h2 class='title comment-form__title'> <i class="fa fa-comments-o"></i> Add new comment</h2> <drupal-render-placeholder callback="comment.lazy_builders:renderForm" arguments="0=node&amp;1=65&amp;2=comment&amp;3=comment" token="CjAkHiB13vKN1b_bf665Z85l9BLJbHaNxdAz1gFUm_U"></drupal-render-placeholder> </section> Mon, 23 Mar 2015 13:31:43 +0000 Hoti 65 at https://tech.feedyourhead.at