Replace incorrectly displayed special chars in bash - bash

I've uploaded a big number of files including their folder structure to my Ubuntu 12.04 LTS Server using WinSCP.
The goal is to access these files in Owncloud.
However, all files that contain special character like German Umlauts cause problems. In Ownclouds view, their name is cut off at the special character and trying to view that folder or file will send you back to the folder root.
Using ls, the special character is always displayed as a question mark, e.g. "Moterschwei?en1.jpg"
What works is manually renaming them through "mv" in the shell. Inserting the special char properly, e.g. "Motorschweißen1.jpg" for this example, does work, but doing this for all of them would take ages.
Using find . -name "?" will not yield any hits.
Is there any way to replace all of those special characters, e.g. with an underscore?

Try the command rename:
rename 'y/\W/_' *
The above command will replace all non alphanumeric characters with _. See http://perldoc.perl.org/perlop.html#Regexp-Quote-Like-Operators and http://perldoc.perl.org/perlre.html#Special-Backtracking-Control-Verbs for the documentation of perl regex expression.

Related

Running command on windows does not allow quotations

when i run a command on windows 10 command line that requires a path as one of its params, it works if the path is NOT inside a quotation, but if a path has a space in it, i need to wrap it inside quotes so that it treats as one single path, but then it complains that the file in that path does not exists.
For example:
C:/PROJECTS/desktopfiles/public/libs/cpdf/win64/cpdf.exe C:/Users/john/Documents/cat.pdf C:/Users/john/Documents/my_dog.pdf -o C:/Users/john/Documents/cat_dog_Merged.pdf
The above works,
the below doesn't (because there is a space in my dog.pdf)
C:/PROJECTS/desktopfiles/public/libs/cpdf/win64/cpdf.exe C:/Users/john/Documents/cat.pdf C:/Users/john/Documents/my dog.pdf -o C:/Users/john/Documents/cat_dog_Merged.pdf
You could try to replace spaces with a question mark. The question mark is a wildcard to match "any single character", which would be a space in your case. Like this: my?dog.pdf. Just make sure that there is no other file matching this pattern. But the system should give you some error message then (which might or might not point to the root of the problem).
Another solution that comes to my mind is a batch file that renames the files in question automatically (replacing spaces with underscores) and renames them back after the pdf merge.

Sed replace unusual file extension arising from gmv

As a result of using gmv on a large nested directory to flatten in, I have a number of duplicate files separated out and with the extensions "._1_" "._2_" etc ( .... ._n_ )
eg "a.pdf.\_1\_"
ie its
a(dot)pdf(dot)(back slash)1(back slash)
as opposed to
a(dot)pdf(dot)1
which I want to reduce it back to "a.pdf"
I tried something like
sed -i .bak "s|.\_1\_||" *
which is usually reliable and doesn't require escape characters. However its giving me
"error: illegal byte sequence"
Grateful for help to fix. This is on Mac OSX terminal. Ideally I'd like a generic solution to fix ._*_ forms where the * varies 1 to 9
There are two challenges here.
How to deal with the duplicate basename (The suffixes '1', '2', ... mostly like added to designate different sections of a single file - may be different pages a PDF, etc. Performing rename that will strip the files may cause some important files to disappear.
How to deal with the "error: illegal byte sequence" which indicate that some special characters (unicode) are part of the file name. Usually ASCII characters with value >= \0xc0, which can not be decoded according to the current local. The fact that the file names are escaped (as per OP "a.pdf.\_1\_" may hint at additional characters, not displayed (assuming this was not added by the OP).
Proposed solution is to rename the file, and place the 'sequence' part, that make the file unique BEFORE the extension, allowing the extension to be used to determine file type.
a.pdf.1 => a.1.pdf
The rename command to perform this task is:
rename 's/(.).pdf.(_._)/$1$2.pdf/' .pdf.__
Adjust the file name list as needed, and use -n to verify before running.
rename -n s/.\_1\_// *.*_1_
works (remove the -n once tested).

Is there any path magic to specify an external drive in Windows from relative path?

Without using slashes (/) or backslashes (\), is there any way I can create a file on an external drive using a relative path on windows?
Or, put less confusingly...
Say I have to append whatever I do to the path:
C:\Application Data\blib\
Is there any way I can do something like:
C:\Application Data\blib\<WINDOWS_SORCERY>external:\wherever\file.txt
I've tried inserting delete characters (\x7f) and backspace characters (\x08), and the reserved characters, but nothing doing. And from what I can tell the reserved characters would only be helpful if the file were being created from within a batch script; it's being created in a Java program.
edit: By 'nothing doing' I mean I get an error message; the program doesn't strip those characters out.

Installing emacs plugins on windows

I already looked through other topics, but I still couldn't find a solution. I'm trying to install "nxhtml" plugin for Emacs in windows 7. I already setup my "HOME" environment variable as "C:\". So, my .emacs.d folder is there, and I put the nxhtml in there and added the following line to my "_emacs.d" file, as the readme says:
(load "C:\.emacs.d\nxhtml\autostart.el")
But it doesn't load.
I also tried putting:
(add-to-list 'load-path "C:\.emacs.d\nxhtml")
(load "autostart.el")
But to no avail... can anyone shed some light here? tnx.
A number of points here:
Firstly, _emacs.d is not a default file name for your init file, ie emacs will not load it automatically. Try ~/.emacs.d/init.el, or ~/.emacs instead.
Secondly, Windows 7 has a feature where it prevents programs from writing to certain system directories, but for backwards compatibility for the many old programs that do this, rather than causing them to fail, it silently redirects the write elsewhere, in an application specific directory. C:\ is one of those directories, so setting your HOME to point there is asking for trouble.
Thirdly, see the other response about backslash being an escape character in Lisp strings.
\ is special in the (double-quote) read syntax for strings, as certain characters take on a new meaning when prefixed by a backslash (e.g. \n is a newline, \t is a tab, and \" is a double-quote character). When the following character does not have any special meaning in conjunction with the backslash, that character is used verbatim, and the backslash is ignored.
"C:\.emacs.d\nxhtml\autostart.el" is actually the string:
C:.emacs.d
xhtml^Gutostart.el
To include a \ in the string you need to write \\
However, although it will understand the backslashes, Emacs is nowadays consistent across all platforms in allowing / as a directory separator1; so just do that instead.
1 and the obsolete directory-sep-char variable has been removed entirely.

Testing "framework" for scripts with nonstandard filenames

Here are many comments on some questions (especially for shell) that say basically one or more of the following:
This will fail on file names that contain spaces, newlines, etc,
This will fail if the file is a symbolic link (or not),
This will fail if the $filaneme is a directory and not regular file,
and so on.
While I understand that every script needs its own testing environment, but
these are some common things for what the script should be immune against.
So, my intention is to write a script what will create some directory hierarchy
with "specially crafted" file names for testing purposes.
The question is: what "special" file names are good for this test?
Currently I have (the script creates files and directories) with:
space in the file name
newline in the file name
file name that starts with one of:
- (like command argument)
# (comment char)
! (command history)
file name that contains one of:
| char (pipe)
() chars
* and ? (wildcards)
file name with unicode characters
all above for the directories
symbolic link to the directory
symbolic link to the file
Any other idea what I shouldn't miss?
What comes to my mind:
quotes in the filename single and double
the $ character at the start
several redirection characters like > < << <<<
the ~ char ($HOME)
the ';' (as command delimiter)
backslash in the filename \
basically, go thru ascii table and test all chars, if you think that you need this :)
Some another comments:
If you want test scripts for the stack-overflow questions, you should create one file with the OP's content (calling as the "basic file")
And the all above "special files" should be symlinks to the above basic file. With this method you can easily modify the content of the files (you need change only one - the basic).
Or, if symlinks not a solution for you use hard-links.
Not directly about special characters in the filenames, but it is good care about:
different case filenames, especially for images like image.jpg image.JPG, same filename only different extension
EDIT: Ideas from the comments:
Very long filenames, lots and lots of files, and very deep directory hierarchies (tripleee)

Resources