Hifidelio-User.de

Das (inoffizielle!) Forum für Hifidelio-User
Aktuelle Zeit: 23.10.2017 15:35

Alle Zeiten sind UTC + 1 Stunde [ Sommerzeit ]




Ein neues Thema erstellen Auf das Thema antworten  [ 28 Beiträge ]  Gehe zu Seite 1, 2  Nächste
Autor Nachricht
BeitragVerfasst: 28.01.2008 18:39 
Offline

Registriert: 04.03.2006 01:02
Beiträge: 39
Wohnort: London UK
I finally decided to do something about the problem with the Linux filenames not being compatible with the Windows format which makes backing up the files via Windows a real pain. So today I learnt Perl and made a Perl script musiclink.pl that will create a sub-directory called .music and in that directory it will create symbolic links to the original files but it will replace any invalid characters in the directory and filenames. You will then be able to back up the .music directory on a Windows PC (or copy to a NAS for example).

To use this script copy it to the Hifidelio import directory for example and then telnet on to the Hifidelio:

> cd /audio/music
> chmod a+x /audio/import/musiclink.pl
> /audio/import/musiclink.pl

You will then end up with a directory called /audio/music/.music with a nice clean version of the files. If a sub-directory does not have any invalid filenames then it will be replaced with a single link. If a sub-directory has any invalid filenames then it will contain links for all of the files in that sub-directory.

You can re-run the script whenever you want as it will first remove the .music directory and then rebuild its contents.

The script currently replaces ?, ", : characters and also if the directory name ends in a full stop. There may be more so please let me know.

Code:
#!/usr/bin/perl

# musiclink.pl
# Hifidelio script for creating symbolic links to music files with invalid Windows names.
#
# Revision 0.4  2008/04/05  Simon Heather
# Fixed checking of sub-directories.
#
# Revision 0.3  2008/01/29  Simon Heather
# Improve performance.
#
# Revision 0.2  2008/01/29  Simon Heather
# Improve performance.
#
# Revision 0.1  2008/01/28  Simon Heather
# Initial working version.
#

use warnings;

sub fix_name
{
   my ($file) = @_;
   
   $file =~ tr/?:"/__'/;
   
   my $newfile = $file;
   if (chop($newfile) eq '.')
   {
      return $newfile . '_';
   }
   
   return $file;
}
   
sub escape_name
{
   my ($file) = @_;
   my $newfile = "'";
   
   while (($index = index($file, "'")) >= 0)
   {
      $newfile = $newfile . substr($file, 0, $index) . "'" . '"' . "'" . '"' . "'";
      $file = substr($file, $index+1, length($file) - ($index+1));
   }
   
   $newfile = $newfile . $file . "'";
   
   return $newfile;
}

my $check_dir;
$check_dir = sub
{
   my ($sourcedir) = @_;
   
   print "checking directory " . $sourcedir . "\n";
   chdir($sourcedir) || die "Unable to chdir to $sourcedir\n";

   my $file;

   my @files = <*>;
   foreach $file (@files)
   {
      if (fix_name($file) ne $file)
      {
         return 1;
      }
      
      if (-d $file)
      {
         my $nextsourcedir = $sourcedir . $file . '/';
         if ($check_dir->($nextsourcedir) > 0)
         {
            return 1;
         }
         chdir($sourcedir);
      }
   }
   
   return 0;
};

my $process_dir;
$process_dir = sub
{
   my ($sourcedir, $destdir) = @_;
   
   chdir($sourcedir) || die "Unable to chdir to $sourcedir\n";
   
   my $file;

   my @files = <*>;
   foreach $file (@files)
   {
      if (-d $file)
      {
         my $newfile = fix_name($file);
         my $nextsourcedir = $sourcedir . $file . '/';
         my $nextdestdir = $destdir . $newfile . '/';

         if ($file ne $newfile or $check_dir->($nextsourcedir) > 0)
         {
            system("mkdir " . escape_name($nextdestdir));
            $process_dir->($nextsourcedir, $nextdestdir);
         }
         else
         {
            $cmd = 'ln -s ' . escape_name($sourcedir . $file) . ' ' . escape_name($destdir . $newfile);
            print $cmd . "\n";
            system($cmd);
         }
         
         chdir($sourcedir);
      }
      else
      {
         $newfile = fix_name($file);
         $cmd = 'ln -s ' . escape_name($sourcedir . $file) . ' ' . escape_name($destdir . $newfile);
         print $cmd . "\n";
         system($cmd);
      }
   }
};

$rootdir = `pwd`;
chomp($rootdir);
$rootdir = $rootdir . '/';
$linkdir = $rootdir . ".music/";

print "running in directory " . $rootdir . "\n";

system("rm -rf " . escape_name($linkdir));
system("mkdir " . escape_name($linkdir));

$process_dir->($rootdir, $linkdir);


Zuletzt geändert von brookheather am 06.04.2008 23:01, insgesamt 6-mal geändert.

Nach oben
 Profil  
 
 Betreff des Beitrags:
BeitragVerfasst: 28.01.2008 22:03 
Offline
Moderator
Benutzeravatar

Registriert: 29.01.2005 00:43
Beiträge: 1750
Wohnort: CH, Bern
Hello
A very good idea! Isn't there any possiblility to "teach" samba to replace certain characters? (On the fly would be "nicer" than doing it before each backup...)

Beside this:
I always get:
Code:
root@hifideliopro:/audio/music# /audio/import/musiclink.pl
bash: /audio/import/musiclink.pl: bad interpreter: Permission denied

I've already tried to change the 1st line of your script to
Code:
#!/opt/perl/bin/perl

...the error message does not change...
Any hints?
Regards, Bergi

_________________
Was war ST-64 :?:


Nach oben
 Profil  
 
 Betreff des Beitrags:
BeitragVerfasst: 28.01.2008 22:07 
Offline
Moderator
Benutzeravatar

Registriert: 29.01.2005 00:43
Beiträge: 1750
Wohnort: CH, Bern
Ah, ok ;-)
There was just a
Code:
chmod 755 /audio/import/musiclink.pl

missing.
Now it is doing something ;-)
I'll report later...
Regards, Bergi

_________________
Was war ST-64 :?:


Nach oben
 Profil  
 
 Betreff des Beitrags:
BeitragVerfasst: 29.01.2008 01:59 
Offline

Registriert: 04.03.2006 01:02
Beiträge: 39
Wohnort: London UK
Thanks - instructions have been updated.


Nach oben
 Profil  
 
 Betreff des Beitrags:
BeitragVerfasst: 29.01.2008 02:01 
Offline

Registriert: 04.03.2006 01:02
Beiträge: 39
Wohnort: London UK
Bergi hat geschrieben:
Isn't there any possiblility to "teach" samba to replace certain characters?


Yes it's a shame that Samba uses such a clumsy way of mangling the names - if only it just replaced invalid characters and preserved the file type then we wouldn't have this problem. I believe Samba would need to be rewritten to achieve this so I created this Perl script instead as I don't fancy changing the Samba code though in principle this could be done.

- Simon.


Nach oben
 Profil  
 
 Betreff des Beitrags:
BeitragVerfasst: 29.01.2008 09:29 
Offline
Moderator
Benutzeravatar

Registriert: 29.01.2005 00:43
Beiträge: 1750
Wohnort: CH, Bern
Hello
Code:
#!/opt/perl/bin/perl

Was still neccessary @ 1st Line of your script.
This is possibly because I am using the (rather buggy) HF version 2.3.19
Regards, Bergi[/quote]

_________________
Was war ST-64 :?:


Nach oben
 Profil  
 
 Betreff des Beitrags:
BeitragVerfasst: 29.01.2008 09:49 
Offline

Registriert: 04.03.2006 01:02
Beiträge: 39
Wohnort: London UK
Thanks. How well did it work - any problems?


Nach oben
 Profil  
 
 Betreff des Beitrags:
BeitragVerfasst: 29.01.2008 11:20 
Offline
Moderator
Benutzeravatar

Registriert: 29.01.2005 00:43
Beiträge: 1750
Wohnort: CH, Bern
brookheather hat geschrieben:
Thanks. How well did it work - any problems?

No, no forther problems, it worked wall and i've found the .music Folder with all the Replaced names.
Do you see a way to do this "on-line" it's not very user-friendly (sorry) to run it before every backup...

Regards, Bergi

_________________
Was war ST-64 :?:


Nach oben
 Profil  
 
 Betreff des Beitrags:
BeitragVerfasst: 29.01.2008 13:02 
Offline

Registriert: 04.03.2006 01:02
Beiträge: 39
Wohnort: London UK
You could add the musiclink script to the startup script /etc/rc.local by adding this code to the end:

Code:
cd /audio/music
/usr/local/bin/musiclink.pl &


(assuming you put the musiclink script in /usr/local/bin). Then whenever you turn on your Hifidelio it will rebuild the .music directory.

- Simon.


Nach oben
 Profil  
 
 Betreff des Beitrags:
BeitragVerfasst: 29.01.2008 14:56 
Offline
Benutzeravatar

Registriert: 16.08.2005 08:58
Beiträge: 216
Wohnort: Heppenheim
I guess this was not written by a perl programmer... Basically the whole fix_name function can be effectively written as single
Code:
tr/?:"/__'/;

statement. Characters like <, >, |, \ and other characters that are forbidden in Windows filenames are not handled either (or does the Hifidelio not create files with those characters?

It is very difficult to do automatic renaming to more similar names in Samba because handling clashes fast is not that easy with complex filenames. Suppose a directory contains the files Test, Test_ and Test?. What do you want to rename Test? to? You definitely want accesses to Test_ and Test to access the correct files.

One more thing the script needs to do is map file names and directories that differ only in case to different files/directories. Windows shares can display all directories TEST, test and Test, but when entering one of them, you always end up in the same one.

However, the script is a good idea and if I find some time I will look into it a little more.

Thomas


Nach oben
 Profil  
 
 Betreff des Beitrags:
BeitragVerfasst: 29.01.2008 14:58 
Offline

Registriert: 04.03.2006 01:02
Beiträge: 39
Wohnort: London UK
Thanks - you are correct I am not a Perl programmer - I just googled for examples of things I wanted to achieve. It would be great if someone could upgrade the script - I am working on a revised version that is more efficient and I will post this later today.

- Simon.


Nach oben
 Profil  
 
 Betreff des Beitrags:
BeitragVerfasst: 29.01.2008 16:00 
Offline
Godfather of Hifidelio
Benutzeravatar

Registriert: 03.02.2005 12:45
Beiträge: 2040
Wohnort: Rheinhessen
tobox hat geschrieben:
It is very difficult to do automatic renaming to more similar names in Samba because handling clashes fast is not that easy with complex filenames. Suppose a directory contains the files Test, Test_ and Test?. What do you want to rename Test? to? You definitely want accesses to Test_ and Test to access the correct files.


This is a problem I thought about some time ago: You could for example renumber the renamed names until they are unique. Inside SAMBA this is even worse: What if somebody else now creates a NEW file with a renumbered name? You are not allowed to change your name, because some client might already have a path on that, so do you want to change the new (and perfectly legal) name?

_________________
Verallgemeinerungen sind generell schlecht.


Nach oben
 Profil  
 
 Betreff des Beitrags:
BeitragVerfasst: 29.01.2008 22:59 
Offline

Registriert: 04.03.2006 01:02
Beiträge: 39
Wohnort: London UK
The version 0.3 script now runs in a few seconds for my library of 6000 songs. I have added the script to /etc/rc.local so the .music backup directory will be rebuilt whenever the player is turned on.

- Simon.


Nach oben
 Profil  
 
 Betreff des Beitrags:
BeitragVerfasst: 30.01.2008 00:06 
Offline
Moderator

Registriert: 27.01.2005 21:11
Beiträge: 2436
Musikuss hat geschrieben:
tobox hat geschrieben:
It is very difficult to do automatic renaming to more similar names in Samba because handling clashes fast is not that easy with complex filenames. Suppose a directory contains the files Test, Test_ and Test?. What do you want to rename Test? to? You definitely want accesses to Test_ and Test to access the correct files.


This is a problem I thought about some time ago: You could for example renumber the renamed names until they are unique. Inside SAMBA this is even worse: What if somebody else now creates a NEW file with a renumbered name? You are not allowed to change your name, because some client might already have a path on that, so do you want to change the new (and perfectly legal) name?

Well, what samba now does is to build an almost human-unreadable hash whenever a filename with forbidden characters arises. Why not do both - replace forbidden characters with a _ *and* adding a (maybe shorter) hash value to the name?


Nach oben
 Profil  
 
 Betreff des Beitrags:
BeitragVerfasst: 30.01.2008 00:19 
Offline

Registriert: 04.03.2006 01:02
Beiträge: 39
Wohnort: London UK
gigi hat geschrieben:
Well, what samba now does is to build an almost human-unreadable hash whenever a filename with forbidden characters arises. Why not do both - replace forbidden characters with a _ *and* adding a (maybe shorter) hash value to the name?


And most importantly preserve the file extension! Losing the .flac (or .m4a) extension on the copied files is probably the biggest problem I have with the way Samba works on the Hifidelio.

- Simon.


Nach oben
 Profil  
 
Beiträge der letzten Zeit anzeigen:  Sortiere nach  
Ein neues Thema erstellen Auf das Thema antworten  [ 28 Beiträge ]  Gehe zu Seite 1, 2  Nächste

Alle Zeiten sind UTC + 1 Stunde [ Sommerzeit ]


Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 1 Gast


Du darfst keine neuen Themen in diesem Forum erstellen.
Du darfst keine Antworten zu Themen in diesem Forum erstellen.
Du darfst deine Beiträge in diesem Forum nicht ändern.
Du darfst deine Beiträge in diesem Forum nicht löschen.

Suche nach:
Gehe zu:  
cron
Powered by phpBB® Forum Software © phpBB Group
Deutsche Übersetzung durch phpBB.de