DSM-G600, DNS-3xx and NSA-220 Hack Forum

Focher · 2007-07-07 17:12:14

Hi,

I have seen a few discussions on certain characters not being supported in file and folder names. My guess is that it is all due to Samba not being configured to support Unicode.

My samba.conf reads:

[ global ]
client code page = 850
character set = ISO8859-1

And my /etc/samba/codepages folder shows:

codepage.850 unicode_map.850 unicode_map.ISO8859-1

In my case, I have files that include Cyrillic characters and these always get converted to underscores. I am using a Mac and its Finder to manipulate the files, but I have Russian active on my machine and a Russian keyboard. My locale shows Russian with:

ru_RU
ru_RU.CP1251
ru_RU.CP866
ru_RU.ISO8859-5
ru_RU.KOI8-R
ru_RU.UTF-8

Of course, writing to a local HFS+ formatted drive works fine as does a USB mounted FAT32 drive.

Any ideas how to get Unicode character support working? Changing the language setting in the EasySearch utility doesn't seem to do it.

transstilben · 2007-07-07 18:37:18

When following the discussion concerning the used character set i get the impression that it has something to do with the samba version (2.x versus 3.x):

see: http://dns323.kood.org/forum/viewtopic. … 18&p=1

Maybe someone can confirm that with the new firmware version 1.04 and the new samba version 3.x we will get rid of the problems mentioned here ?

Focher · 2007-07-08 10:53:13

From the Samba HOWTO at http://de.samba.org/samba/docs/man/Samb … icode.html

"As of Samba-3, Samba can (and will) talk Unicode over the wire."

"unix charset - This is the charset used internally by your operating system. The default is UTF-8, which is fine for most systems and covers all characters in all languages. The default in previous Samba releases was to save filenames in the encoding of the clients for example, CP850 for Western European countries.

display charset - This is the charset Samba uses to print messages on your screen. It should generally be the same as the unix charset.

dos charset - This is the charset Samba uses when communicating with DOS and Windows 9x/Me clients. It will talk Unicode to all newer clients. The default depends on the charsets you have installed on your system. Run testparm -v | grep "dos charset" to see what the default is on your system. "

In my case, I am accessing from OS X and saw the following in the HOWTO:

"Although Mac OS X uses UTF-8 as its encoding method for filenames, it uses an extended UTF-8 specification that Samba cannot handle, so UTF-8 locale is not available for Mac OS X."

This concerns me a bit that even Samba 3 won't solve the problem.

noodle · 2007-07-16 20:43:27

I tried firmware 1.04 beta, samba 3.x is very good on Unicode support. What I did is: create a fun_plug start script to stop default samba process and start samba with my customized configuration file. BTW, it's possible to set one folder has different permissions for different user/group (which I don't think D-Link web GUI support it). Here is the files:

Customized samba startup script
/mnt/HD_a2/fun_plug.d/start/samba.sh
#!/bin/sh

if [ -f ${ETCDIR}/samba/smb.conf ]; then
kill `${BINDIR}/pidof smbd`
kill `${BINDIR}/pidof nmbd`
# If you want to use customized smbpasswd file as well, uncomment next line
# smbd -D -s ${ETCDIR}/samba/smb.conf --smb-passwd=${ETCDIR}/samba/smbpasswd
# If you want to use customized smbpasswd file as well, comment next line
smbd -D -s ${ETCDIR}/samba/smb.conf
nmbd -D -s ${ETCDIR}/samba/smb.conf
fi

Customized samba configuration file
/mnt/HD_a2/fun_plug.d/etc/samba/smb.conf
[ global ]
interfaces = egiga0
# Note: this is impotent to make samba work with Unicode
unix charset = UTF8
workgroup = WORKGROUPO
netbios name = <Your server name here>
server string = <Your server description here>
hosts allow =
hosts deny =
security = SHARE
# I'm still working on security set to user.
#security = user
encrypt passwords = yes
#If you want to customize smbusers file, uncomment next line
#username map = /mnt/HD_a2/fun_plug.d/etc/samba/smbusers
max log size = 0
socket options = TCP_NODELAY SO_RCVBUF=65536 SO_SNDBUF=65536
max xmit = 65535
# You can change mask to not allow new created file wildly permitted
create mask = 0666
directory mask = 0777
force create mode = 0666
force directory mode = 0777
load printers = Yes
printcap name = /usr/local/LPRng/etc/printcap
min print space = 2000
max print jobs = 1000
printing = lprng
print command = /usr/local/LPRng/bin/lpr -P%p -r %s
lpq command = /usr/local/LPRng/bin/lpq -P%p
lprm command = /usr/local/LPRng/bin/lprm -P%p %j
lppause command = /usr/local/LPRng/sbin/lpc hold %p %j
lpresume command = /usr/local/LPRng/sbin/lpc release %p %j
queuepause command = /usr/local/LPRng/sbin/lpc -P%p stop
queueresume command = /usr/local/LPRng/sbin/lpc -P%p start
use sendfile =yes

[ web_page ]
comment = Enter Our Web Page Setting
path = /mnt/web_page
valid users =
read only = yes
guest ok = yes

[printers]
path = /mnt/HD_a2/.lpd
guest ok = Yes
printable = Yes
use client driver = Yes
browseable = No

[ Volume_1 ]
comment =
path = /mnt/HD_a2
valid users =
read only = no
guest ok = yes

[ Volume_2 ]
comment =
path = /mnt/HD_b2
valid users =
read only = no
guest ok = yes

Last edited by noodle (2007-07-18 20:00:49)

ldomaigne · 2007-07-17 14:15:49

Hi,

I had real troubles with non english characters in filenames, see:
http://dns323.kood.org/forum/t651-probl … linux.html

The solution to my problem was to upgrade to Firmware 1.04 (beta). It is based on Samba 3.x, which uses UTF-8 as internal encoding for the filenames, removing all the pains.

Accordingly to the D-Link German support, the official FW 1.04 should be released "pretty soon".

Cheers,
Loïc.

fonz · 2007-07-17 15:25:00

ldomaigne wrote:
Accordingly to the D-Link German support, the official FW 1.04 should be released "pretty soon".

Is D-Link Germany going to skip 1.03? On their website, current FW is still 1.02B114.

Focher · 2007-07-18 16:36:16

Is anyone using the 1.04 beta firmware with a Mac client? Are extended characters supported through Samba 3's Unicode support?

From my research on Samba 3, it seems there is no problem with Unicode support anymore on Windows or *nix, but the slightly different UTF-8 encoding from OS X gives the impression that Unicode support through an OS X client will still have problems.

eko · 2007-07-18 17:58:15

Focher wrote:
Is anyone using the 1.04 beta firmware with a Mac client? Are extended characters supported through Samba 3's Unicode support?

From my research on Samba 3, it seems there is no problem with Unicode support anymore on Windows or *nix, but the slightly different UTF-8 encoding from OS X gives the impression that Unicode support through an OS X client will still have problems.

yes Focher, i've got the DNS-323 (fun_plugged) with firmware 1.04beta (from that russian website) on the LAN together with a mac with system OS X 10.5 (leopard from WWDC'07). is that a similar setup to yours or are you "asking before buying"?

i read in the release notes that samba was updated to v3, so i tried verifying it with the command "samba -v" from a SSH session from the mac but it didn't recognize the command (can't remember the exact returned message but i can find out). what am i supposed to type if not "samba -v"

i can confirm SAMBA is operational, and i can write files with char a to z, 0 to 9 and the special ones such as - & _() i have not tried other, if you need me to test i'll be happy to help !

so long, g33ks !

ldomaigne · 2007-07-19 10:16:15

fonz wrote:
ldomaigne wrote:
Accordingly to the D-Link German support, the official FW 1.04 should be released "pretty soon".
Is D-Link Germany going to skip 1.03? On their website, current FW is still 1.02B114.

Well, I raised a similar question to D-Link Germany... But I didn't got any answer. For sure, they have the 1.03 FW, but for some reasons they don't release it. Either someone isn't doing her/his job correctly and forgot to put the 1.03 FW on the FTP server. Or, they don't want to...

To be continued...

Cheers,
Loïc.

ldomaigne · 2007-07-19 10:24:45

Focher wrote:
Is anyone using the 1.04 beta firmware with a Mac client? Are extended characters supported through Samba 3's Unicode support?

Yes. That's exactly the point of UTF-8.

Focher wrote:
From my research on Samba 3, it seems there is no problem with Unicode support anymore on Windows or *nix, but the slightly different UTF-8 encoding from OS X gives the impression that Unicode support through an OS X client will still have problems.

Wouldn't it defeat the purpose of the UTF-8 standard, if the Operating System vendors implemented their own flavor of it? So in theory, that should work fine... But as you know, practice might look different.

Cheers,
Loïc.

frodo · 2007-07-20 00:38:51

Seems like MacOS X needs UTF-8-MAC in the smb.conf file.

Read more here:
http://lists.samba.org/archive/samba-te … 14862.html

This seems to have been added to the latest versions of Samba.

ka3ak · 2007-07-20 01:20:00

Hi there! ....
Didn't want to start a new topic .... but I have a question :

I got some files that contain Cyrillic charcters in them and they are completely invisible from let's say Windows Explorer when the folder is mapped as a netwrok Drive (it got me there as I was thinking that my music files are gone !!! ).

Through FTP session and Telnet they are visible but the names are not shown correctly (bunch of wierd letters instead). I tried to change the locale with EasySearch Utility to Cyrillic but it still shows me crap through FTP or Telnet. the content of smb.conf is the same as Foscher had :

[ global ]
client code page = 850
character set = ISO8859-1

And my /etc/samba/codepages folder shows:

codepage.850 unicode_map.850 unicode_map.ISO8859-1

What should I do to make those name display correctly in Explorer, FTP session and Telnet ?

Updated

I have changed the "character set" entry in smb.conf to CP1251 and rebooted the unit and lo and behold ... the damn names started appearing as normal BUT only in FTP or Telnet session. If I go to My computer ->Mapped disks (y) .... it shows me nothing in the directory where files with Cyrillic names are.

Last edited by ka3ak (2007-07-20 04:23:11)

CurtisL · 2007-07-20 05:35:42

I updated my firmware to the 1.04 beta, and implemented the samba.sh script on this topic, and I was able to see my special characters in telnet, ftp and through my fun_plugged lighttp. However, Windows (which had previously shown them correctly) was showing junk.
I renamed the files using Windows explorer (to re-add the special characters), and everything was synched.
Pain in the butt to go through and re-name everything again, but its nice to finally put that annoyance to bed.

Edit:
Oh! And ... if your telnet or FTP client isn't showing them correctly, make sure you've set it to use UTF-8 as the character set. PuTTY is a great free client that works ...

HTH,
Curtis

Last edited by CurtisL (2007-07-20 05:36:56)

Focher · 2007-07-20 11:37:21

ka3ak wrote:
Hi there! ....
I got some files that contain Cyrillic charcters in them and they are completely invisible from let's say Windows Explorer when the folder is mapped as a netwrok Drive (it got me there as I was thinking that my music files are gone !!! ).

Cyrillic is exactly the character set that I have problems with too. Samba v3 (in the 1.04 firmware) will fix the problem.

ka3ak · 2007-07-20 20:54:33

Focher

I updated Firmware to 1.4 ..... now it shows the files and folders with Cyrillic letters (before it showed just empty folder) BUT they are all screwed up.... same wierd characters. The only thing that changed is that now it shows that there is actually something in the folder. The only solution I see is just to rename damn files.....

I also got problems with FTP server after update to 1.4.... For some reason I can only connect with Administrator( has R/W rights in root) login to root of FTP.... Any other userID I create (they have access not to the root folder but to sub-folders and have Read-only privileges) are useless ... Pure-FTP just closes connection after entering the password. Here is the log :

*** CuteFTP 8.0 - build May 31 2007 ***

STATUS:> [7/20/2007 11:48:13 AM] Getting listing ""...
STATUS:> [7/20/2007 11:48:13 AM] Resolving host name xxx.serveftp.com...
STATUS:> [7/20/2007 11:48:14 AM] Host name xxx.serveftp.com resolved: ip = xxxxxxxx
STATUS:> [7/20/2007 11:48:14 AM] Connecting to FTP server... xxx.serveftp.com:21 (ip = xxxxxxx)...
STATUS:> [7/20/2007 11:48:14 AM] Socket connected. Waiting for welcome message...
[7/20/2007 11:48:14 AM] 220---------- Welcome to Pure-FTPd ----------
220-You are user number 1 of 5 allowed.
220-Local time is now 11:48. Server port: 21.
220 You will be disconnected after 2 minutes of inactivity.
STATUS:> [7/20/2007 11:48:14 AM] Connected. Authenticating...
COMMAND:> [7/20/2007 11:48:14 AM] USER movies
[7/20/2007 11:48:14 AM] 331 User movies OK. Password required
COMMAND:> [7/20/2007 11:48:14 AM] PASS *****
[7/20/2007 11:48:14 AM] 230 OK. Current restricted directory is /
STATUS:> [7/20/2007 11:48:14 AM] Login successful.
COMMAND:> [7/20/2007 11:48:14 AM] PWD
[7/20/2007 11:48:14 AM] 257 "/" is your current location
STATUS:> [7/20/2007 11:48:14 AM] Home directory: /
COMMAND:> [7/20/2007 11:48:14 AM] FEAT
[7/20/2007 11:48:14 AM] Informational Message Only:
211-Extensions supported:
   EPRT
   IDLE
   MDTM
   SIZE
   REST STREAM
   MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*;
   MLSD
   TVFS
   ESTP
   PASV
   EPSV
   SPSV
   ESTA
   UTF8
211 End.
STATUS:> [7/20/2007 11:48:14 AM] This site supports features.
STATUS:> [7/20/2007 11:48:14 AM] This site supports SIZE.
STATUS:> [7/20/2007 11:48:14 AM] This site can resume broken downloads.
COMMAND:> [7/20/2007 11:48:14 AM] REST 0
[7/20/2007 11:48:14 AM] 350 Restarting at 0
COMMAND:> [7/20/2007 11:48:14 AM] PASV
[7/20/2007 11:48:14 AM] 227 Entering Passive Mode (68,145,180,59,87,76)
COMMAND:> [7/20/2007 11:48:14 AM] LIST
STATUS:> [7/20/2007 11:48:14 AM] Connecting FTP data socket... xx.xxx.xxx.xx:22348...
[7/20/2007 11:48:14 AM] 150 Accepted data connection
ERROR:>    [7/20/2007 11:48:14 AM] Control connection closed. <- THAT IS WHERE IT DROPS CONNECTION
STATUS:> [7/20/2007 11:48:14 AM] This site can resume broken downloads.

Looks like it has problems with PASv(passive redirection). It tries to connect to port 22348 ... In other FTP servers there is an option for IP binding and Passive port ranges where you can setup redirection in PASV mode.... but here there is nothig similar in Pure-ftp.conf... Or maybe I am just blind.... Well , I'm downgrading to 1.03...

Last edited by ka3ak (2007-07-20 20:56:43)

Focher · 2007-07-21 15:45:25

I stuck the 1.04 beta firmware on and the Samba is at v3 (smbd -V) but the codepage is still set to CP850. If I change the language to Cyrillic (which isn't a language...), the codepage changes to a bunch of Cyrillic special ones. This allows Samba shares to see the Cyrillic characters but not in Telnet.

ka3ak · 2007-07-22 03:23:31

Focher wrote:
I stuck the 1.04 beta firmware on and the Samba is at v3 (smbd -V) but the codepage is still set to CP850. If I change the language to Cyrillic (which isn't a language...), the codepage changes to a bunch of Cyrillic special ones. This allows Samba shares to see the Cyrillic characters but not in Telnet.

Thats exactly what happens in my case...... I have two outcomes - either with 1.3 I don't have anything in the folder (unit desn't list the files or folders with Russian characters) OR I have the list of files and folders but with Cyrillic special ones not the normal ones. I hope it is going to be fixed when 1.4 is released Fingers crossed.

P.S. Did you encounter FTP problem that I have when upgraded to 1.4 ? Or is it I am doing something wrong... My guess is that I probably will have to delete all the users created before on FW 1.3 and then upgrade to 1.4, to prevent any interfereance with older accounts.

Focher · 2007-07-22 05:20:12

The characters definitely do not display in FTP or Telnet. That could be because my terminal client isn't support UTF8, but I would have to look further into that.

I tried using noodle's samba.sh (with the minor change in smb.conf to "UTF-8-MAC" instead of "UTF8) but it also did not work. I copied my settings from the Mac's smb.conf file to make sure it matched. The one difference is that it does seem to write the character information because if I try to create another file with the same characters, it says a file with that name already exists. The Cyrillic characters still display as some Latin character equivalent though.

This is all with the 1.04 beta firmware.

CurtisL · 2007-07-24 07:34:49

PuTTY supports UTF-8
http://www.chiark.greenend.org.uk/~sgta … nload.html

Firmware 1.04 shows me:
BusyBox v1.6.1 (2007-07-15 20:22:06 CEST) Built-in shell (ash)
Enter 'help' for a list of built-in commands.

/ # cd /mnt/HD_a2/Music/MP3/
/mnt/HD_a2/Music/MP3 # ls -lad Bj*
drwxrwxrwx 4 root root 4096 Jul 18 22:04 Björk
/mnt/HD_a2/Music/MP3 # cd Björk/
/mnt/HD_a2/Music/MP3/Björk # ls -la
drwxrwxrwx 4 root root 4096 Jul 18 22:04 .
drwxrwxrwx 503 root root 12288 Jul 23 21:37 ..
drwxrwxrwx 2 root root 4096 Jul 18 22:03 [2004] Medúlla
drwxrwxrwx 2 root root 4096 Jul 18 22:03 [2007] Volta

With Firmware 1.03, I would get something like `Bj rk` and `Med lla`.

DSM-G600, DNS-3xx and NSA-220 Hack Forum

Announcement

#1 2007-07-07 17:12:14

Unicode through Samba / SMB mount

#2 2007-07-07 18:37:18

Re: Unicode through Samba / SMB mount

#3 2007-07-08 10:53:13

Re: Unicode through Samba / SMB mount

#4 2007-07-16 20:43:27

Re: Unicode through Samba / SMB mount

#5 2007-07-17 14:15:49

Re: Unicode through Samba / SMB mount

#6 2007-07-17 15:25:00

Re: Unicode through Samba / SMB mount

ldomaigne wrote:

#7 2007-07-18 16:36:16

Re: Unicode through Samba / SMB mount

#8 2007-07-18 17:58:15

Re: Unicode through Samba / SMB mount

Focher wrote:

#9 2007-07-19 10:16:15

Re: Unicode through Samba / SMB mount

fonz wrote:

ldomaigne wrote:

#10 2007-07-19 10:24:45

Re: Unicode through Samba / SMB mount

Focher wrote:

Focher wrote:

#11 2007-07-20 00:38:51

Re: Unicode through Samba / SMB mount

#12 2007-07-20 01:20:00

Re: Unicode through Samba / SMB mount

#13 2007-07-20 05:35:42

Re: Unicode through Samba / SMB mount

#14 2007-07-20 11:37:21

Re: Unicode through Samba / SMB mount

ka3ak wrote:

#15 2007-07-20 20:54:33

Re: Unicode through Samba / SMB mount

#16 2007-07-21 15:45:25

Re: Unicode through Samba / SMB mount

#17 2007-07-22 03:23:31

Re: Unicode through Samba / SMB mount

Focher wrote:

#18 2007-07-22 05:20:12

Re: Unicode through Samba / SMB mount

#19 2007-07-24 07:34:49

Re: Unicode through Samba / SMB mount

Board footer