This is artificial psychological limit. Sometimes when you're actively coding, it is quite sad to loose even 5 minutes of work, since productivity is not constant. This is why 15 minutes was chosen as something that is not too critical. There is no other real reason behind this limit other than how I feel it.But seriously, what are you doing that you can't lose more than 15 minutes of? Couldn't it be even 20 minutes, or a half-hour or an hour with 15 minute snapshots only on the ssds (yes, I know the raid0 factor, but the question still applies)?
As an (relatively) early adopter I'm fine using experimental stuff with extra safeties like backups (hey, I've used it even without those while back:)). I fully acknowledge what is current state of BTRFS and want to help make it even better by stressing issues that me and other users encounter, searching for solutions, etc.And perhaps more importantly for your data, btrfs is still considered "stabilizing, not fully stable and mature". Use without backups is highly discouraged, but I'd suggest that btrfs in its current state might not be what you're looking for if you can't deal with loss of more than 15 minutes worth of changes anyway. Be that as it may... Btrfs is definitely not yet optimized. In many cases it reads or writes only one device at a time, for instance, even in RaidN configuration. And there are definitely snapshot scaling issues altho at your newer 500 snapshots total that shouldn't be /too/ bad.
Yes, I'm leaning toward earning new hardware right now, fortunately, laptop allows me to insert 2 x mSATA + 2 x 2.5 SATA drives, so I have exactly 2.5 SATA slot free.Dealing with reality, regardless of how or why, you currently have a situation of intolerably slow receives that needs addressed. From a practical perspective you said an ssd for backups is ridiculous and I can't disagree, but there's another "throw hardware at it" solution that might be a bit more reasonable... Spinning rust hard drives are cheap. What about getting another one, and alternating your backup receives between them? That would halve the load to one every thirty minutes, without changing your 15-minute snapshot and backup policy at all. =:^) So that gives you two choices for halving the load to the spinning rust. Either decide you really can live with half-hour loss of data, or throw only a relatively small amount of money (well, as long as you have room to plug in another sata device anyway, otherwise...) at it for a second backup device, and alternate between them.
I'm not coding C/C++, so my capabilities to improve BTRFS itself are limited, but I'm always trying to find the reason and fix it instead of living with workarounds forever.OTOH, since you mentioned possible coding, optimization might not be a bad thing, if you're willing to put in the time necessary to get up to speed with the code and can work with the other devs in terms of timing, etc. But that will definitely take significant time even if you do it, and the alternating backup solution can be put to use as soon as you can get another device plugged in and setup. =:^)
I'll play with Seekwatcher and optimizing snapshots deletion and will post an update afterwards.
Sincerely, Nazar Mokrynskyi github.com/nazar-pc Skype: nazar-pc Diaspora: nazarpc@xxxxxxxxxxxxxxxxxxxxxxx Tox: A9D95C9AA5F7A3ED75D83D0292E22ACE84BA40E912185939414475AF28FD2B2A5C8EF5261249 On 17.03.16 09:00, Duncan wrote:
Nazar Mokrynskyi posted on Wed, 16 Mar 2016 05:37:02 +0200 as excerpted:I'm not sure what you mean exactly by searching. My first SSD died during waking up from suspend mode, it worked perfectly till last moment. It was not used for critical data at that time, but now I understand clearly that SSD failure can happen at any time. Having RAID0 of 2 SSDs it 2 times more risky, so I'm not ready to lose anything beyond 15 minutes threshold. I'd rather end up having another HDD purely for backup purposes.I understand the raid0 N times the danger part, which is why I only ever used raid0 on stuff like the distro packages cache that I could easily redownload from the net, here. But seriously, what are you doing that you can't lose more than 15 minutes of? Couldn't it be even 20 minutes, or a half-hour or an hour with 15 minute snapshots only on the ssds (yes, I know the raid0 factor, but the question still applies)? What /would/ you do if you lost a whole hour's worth of work? Surely you could duplicate it in the next hour? Or are you doing securities trading or something, where you /can't/ recover work at all, because by then the market and your world have moved on? But in that case... And perhaps more importantly for your data, btrfs is still considered "stabilizing, not fully stable and mature". Use without backups is highly discouraged, but I'd suggest that btrfs in its current state might not be what you're looking for if you can't deal with loss of more than 15 minutes worth of changes anyway. Be that as it may... Btrfs is definitely not yet optimized. In many cases it reads or writes only one device at a time, for instance, even in RaidN configuration. And there are definitely snapshot scaling issues altho at your newer 500 snapshots total that shouldn't be /too/ bad. Dealing with reality, regardless of how or why, you currently have a situation of intolerably slow receives that needs addressed. From a practical perspective you said an ssd for backups is ridiculous and I can't disagree, but there's another "throw hardware at it" solution that might be a bit more reasonable... Spinning rust hard drives are cheap. What about getting another one, and alternating your backup receives between them? That would halve the load to one every thirty minutes, without changing your 15-minute snapshot and backup policy at all. =:^) So that gives you two choices for halving the load to the spinning rust. Either decide you really can live with half-hour loss of data, or throw only a relatively small amount of money (well, as long as you have room to plug in another sata device anyway, otherwise...) at it for a second backup device, and alternate between them. OTOH, since you mentioned possible coding, optimization might not be a bad thing, if you're willing to put in the time necessary to get up to speed with the code and can work with the other devs in terms of timing, etc. But that will definitely take significant time even if you do it, and the alternating backup solution can be put to use as soon as you can get another device plugged in and setup. =:^)
Attachment:
smime.p7s
Description: Кріптографічний підпис S/MIME
