• Bitrot and atomic COWs: Inside “next-gen” filesystems

    by  • January 16, 2014 • Tech • 0 Comments

    Open_hard-driveA bit on the technical side, but still an interesting article on the future of filesystems.

    Most people don’t care much about their filesystems. But at the end of the day, the filesystem is probably the single most important part of an operating system. A kernel bug might mean the loss of whatever you’re working on right now, but a filesystem bug could wipe out everything you’ve ever done… and it could do so in ways most people never imagine.

    Sound too theoretical to make you care about filesystems? Let’s talk about “bitrot,” the silent corruption of data on disk or tape. One at a time, year by year, a random bit here or there gets flipped. If you have a malfunctioning drive or controller—or a loose/faulty cable—a lot of bits might get flipped. Bitrot is a real thing, and it affects you more than you probably realize. The JPEG that ended in blocky weirdness halfway down? Bitrot. The MP3 that startled you with a violent CHIRP!, and you wondered if it had always done that? No, it probably hadn’t—blame bitrot. The video with a bright green block in one corner followed by several seconds of weird rainbowy blocky stuff before it cleared up again? Bitrot.

    The worst thing is that backups won’t save you from bitrot. The next backup will cheerfully back up the corrupted data, replacing your last good backup with the bad one. Before long, you’ll have rotated through all of your backups (if you even have multiple backups), and the uncorrupted original is now gone for good.

    Contrary to popular belief, conventional RAID won’t help with bitrot, either. “But my raid5 array has parity and can reconstruct the missing data!” you might say. That only works if a drive completely and cleanly fails. If the drive instead starts spewing corrupted data, the array may or may not notice the corruption (most arrays don’t check parity by default on every read). Even if it does notice… all the array knows is that something in the stripe is bad; it has no way of knowing which drive returned bad data—and therefore which one to rebuild from parity (or whether the parity block itself was corrupt).

    What might save your data, however, is a “next-gen” filesystem.

    Read the full article @ ars technica.

    About

    Michael Sauers is currently the Technology Innovation Librarian for the Nebraska Library Commission in Lincoln, Nebraska and has been training librarians in technology for more than 15 years. He has also been a public library trustee, a bookstore manager for a library friends group, a reference librarian, serials cataloger, technology consultant, and bookseller. He earned his MLS in 1995 from the University at Albany’s School of Information Science and Policy. Michael’s twelfth book, Google Search Secrets (w/ Christa Burns) was published October 2013 and has two more books on the way. He has also written dozens of articles for various journals and magazines. In his spare time he blogs at travelinlibrarian.info, runs Web sites for authors and historical societies, takes many, many photos, and reads more than 100 books a year.

    http://www.travelinlibrarian.info/

    Leave a Reply

    Your email address will not be published. Required fields are marked *