DSM-G600, DNS-3xx and NSA-220 Hack Forum

Unfortunately no one can be told what fun_plug is - you have to see it for yourself.

You are not logged in.

Announcement

#1 2008-11-07 10:39:29

puterboy
Member
Registered: 2008-09-18
Posts: 306

WARNING: EXT3 still broken/dropping files in 1.05 (at least with nfsd)

I know that ext3 was considered "broken" in earlier FW versions due to "missing files" which resulted in D-Link removing ext3 from firmwares 1.03, 1.04 and 1.05. I'm not sure what happened then but I assume that various files "disappeared" when accessed using the default samba sharing.

However, several people, including myself thought that maybe it was fixed with Firmware 1.05 and have been reporting it working ok (see http://dns323.kood.org/forum/t2225-Anyo … S-323.html)

Well, after some extensive testing and troubleshooting, I have run into at least one case where Ext3 is severly and repeatedly broken.
Specifically, when used in conjunction with kernel space nfsd (not unfs), random directories and files become unaccessible at random times.
This happens when using a stock 2.6.12.6 kernel recompiled to support the kernel space nfsd server (either internally or as a module).
Dropped files seem to be triggered when there is extensive accessing of the directory tree over nfs such as when doing a recursive 'find' or repeated directory 'ls' operations. This problem DOES NOT occur if the partition is mounted as ext2.

The files and directories do not actually disappear from the dns-323, though their (temporary) absence from file listings can result in severe database corruption as I found out the hard way with BackupPC.

Indeed, you can always get the files to reappear over NFS by doing one of the following:
- Restarting the nfs server
- Listing the same directory on the server (which then recovers it on the remote client).

Again:
- The data and Ext3 filesystem on the dns-323 itself never disappear or get corrupted (except as a side effect of wrong nfs directory results)
- The problem is limited to Ext3 -- Ext2 works fine
- User spaced nfsd (unfsd) works fine
- The missing file/directory listings seem to correlate with heavy directory access and are restored by accessing the same directory from the dns-323.

This behavior suggests to my layman's brain that there is something wrong with how ext3 is caching (or marking as cached) directory/file lookups and sharing this information with other kernel processes requiring access to the filesystem directory.

I wonder whether this problem also affects other network file sharing protocols such as Samba (I only tested Samba briefly and I did not encounter any problems). I suspect that if the problem occurs with Samba it must be a lot rarer since it took a while for D-link to even see the problem vs. with NFS where the problem occurs regularly.

It seems like as long as you stay on the native ext3 filesystem, that everything is ok, but I can't guarantee it.

In conclusion, I would strongly suggest that people avoid using Ext3 given the combination of previous issues, D-links removal of the option, and the severe issues encountered with NFS -- because with so much evidence leaning towards ext3 being broken, it probably does not pay to take the risk that your data will be corrupted due to some othe side effect of this bug (though on the native ext3 system, it seems pretty stable).

If there are any kernel hackers out there, perhaps you could help point me in the direction of a patch or solution (though I spent a lot of time googling and couldn't find anything specific).

D-LINK - if you are reading this thread, can you throw any more light on the issue?
CH3SNAS users - can you verify whether or not you get the same problem on your machines with ext3 + nfsd? (my understanding is that the ch3snas comes with a nfsd-enabled kernel. If the problem doesn't occur there, maybe we could figure out what differences in the kernel patches or versions account for this difference.

Offline

 

#2 2008-11-07 11:01:30

fonz
Member / Developer
From: Berlin
Registered: 2007-02-06
Posts: 1716
Website

Re: WARNING: EXT3 still broken/dropping files in 1.05 (at least with nfsd)

puterboy wrote:

If there are any kernel hackers out there, perhaps you could help point me in the direction of a patch or solution (though I spent a lot of time googling and couldn't find anything specific).

Considering the other problems the D-Link kernel has, and the old version of it, I don't think anyone is seriously interesting in fixing any bugs in it.

Offline

 

#3 2008-11-07 11:07:23

puterboy
Member
Registered: 2008-09-18
Posts: 306

Re: WARNING: EXT3 still broken/dropping files in 1.05 (at least with nfsd)

I understand.
I wish I could try a later kernel (e.g. 2.6.25) but as you pointed out, I can't with HW revB1.

If you have the time/interest, I would be very curious to know the following:
1. Can you reproduce the problem with 2.6.12.6 on your box?
2. Does the problem go away with 2.6.25?

Again you trigger the behavior by repeatedly listing/finding a series of large directories.

If it goes away with 2.6.25, I might be tempted to work harder to find a way to get it to work on my machine.

Offline

 

#4 2008-11-07 11:11:44

fonz
Member / Developer
From: Berlin
Registered: 2007-02-06
Posts: 1716
Website

Re: WARNING: EXT3 still broken/dropping files in 1.05 (at least with nfsd)

puterboy wrote:

I understand.
I wish I could try a later kernel (e.g. 2.6.25) but as you pointed out, I can't with HW revB1.

The fix for B1 is in the orion.git tree since Oct 22:

http://git.kernel.org/?p=linux/kernel/g … 19490399ce

Offline

 

#5 2008-11-07 14:29:33

fordem
Member
Registered: 2007-01-26
Posts: 1938

Re: WARNING: EXT3 still broken/dropping files in 1.05 (at least with nfsd)

Given the fact that you first reported this "randomly dropping files" as occuring with nsfd - should you not attempt to isolate the problem to one or the other?

Offline

 

#6 2008-11-07 17:09:32

puterboy
Member
Registered: 2008-09-18
Posts: 306

Re: WARNING: EXT3 still broken/dropping files in 1.05 (at least with nfsd)

fordem wrote:

Given the fact that you first reported this "randomly dropping files" as occuring with nsfd - should you not attempt to isolate the problem to one or the other?

I wish I could but all I can say is that so far I can only replicate the problem with ext3 + nfsd. Given the history of similar ext3 problems with Samba, the fact that ext2 works fine, and the severity of even a whiff of filesystem corruption, the best I can do is to warn people that the potential danger of filesystem damage probably outweighs the advantages of journaling unless you are very not using nfsd and are near 100% sure that nfsd is the problem.

In other words, when it comes to filesystem integrity, the burden of proof I believe will be to show that ext3 is fine. I don't know what else I can do to take this further other than to keep playing with it.

Offline

 

#7 2008-11-07 17:14:58

puterboy
Member
Registered: 2008-09-18
Posts: 306

Re: WARNING: EXT3 still broken/dropping files in 1.05 (at least with nfsd)

fonz wrote:

puterboy wrote:

I understand.
I wish I could try a later kernel (e.g. 2.6.25) but as you pointed out, I can't with HW revB1.

The fix for B1 is in the orion.git tree since Oct 22:

http://git.kernel.org/?p=linux/kernel/g … 19490399ce

Fonz 2 questions:
- Is there a simple way to download a git kernel without having to download the multi-hundred megabyte tree? I tried to read up on git but wasn't able to figure out how to just download a single snapshot.

- If I use the 2.6.25 to reload the stock initrd, will I lose any of the default functionality? (e.g., fan control or other dns-323 specific functionality)
  Or are there (good) hacks to the 2.6.12 kernel that I will be giving up by moving to 2.6.25?

Offline

 

#8 2008-11-07 17:28:15

fonz
Member / Developer
From: Berlin
Registered: 2007-02-06
Posts: 1716
Website

Re: WARNING: EXT3 still broken/dropping files in 1.05 (at least with nfsd)

puterboy wrote:

- Is there a simple way to download a git kernel without having to download the multi-hundred megabyte tree? I tried to read up on git but wasn't able to figure out how to just download a single snapshot.

no idea. After git-clone, it's around 600MB here.

puterboy wrote:

- If I use the 2.6.25 to reload the stock initrd, will I lose any of the default functionality? (e.g., fan control or other dns-323 specific functionality)
  Or are there (good) hacks to the 2.6.12 kernel that I will be giving up by moving to 2.6.25?

Nothing will work, or it won't boot at all. fan, rtc, spindown, ethernet all use non-standard interfaces.

Offline

 

Board footer

Powered by PunBB
© Copyright 2002–2010 PunBB