We can specify both of flag and perm at os.OpenFile.
They have really similar options, O_APPEND and ModeAppend. What's the difference between them?
f, _ := os.OpenFile("access.log", os.O_APPEND|os.O_CREATE, os.ModeAppend|0644)
The flag specify the flags used on the system call to open the file while perm sets the File mode on the file. The file mode includes the permissions and type of file eg. symlink, directory, etc...
os.O_APPEND tells the underlying OS that all the write calls you do on that file handler should always append to the file so you don't need to set the offset to write on the correct part of the file.
ModeAppend sets the file mode to be append. This means that the this file can only be modified by appending to it, not by rewriting the file contents. The specifics of this depends on the OS and file system you are using. I believe Plan 9, implements it by ignoring the offset on any write call to the file and always appending to it, while in linux it means that the file can only be open for writing in append mode. I think that on most linux distros you need to be root to set the file mode to append.
In 99.99% of cases you just want to use perm to set the file permissions rwx. In your case if you want to open a file and append to it you should use:
// os.O_WRONLY tells the computer you are only going to writo to the file, not read
// os.O_CREATE tells the computer to create the file if it doesn't exist
// os.O_APPEND tells the computer to append to the end of the file instead of overwritting or truncating it
f, err := os.OpenFile("access.log", os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0644)
You might have only ignore the return error on os.OpenFile to put the example online, but you should get used to always checking for errors. You have no idea how many users ran into trouble when starting with go because they ignore the errors. Sometimes is something stupid and easy to fix like a typo, but if you ignore the error you don't know what the issue is.
You can read more about the append file mode here.
Related
Only just now starting with golang, with only a small amount of programming experience before this. I'm trying to create a script that will summarize certain things from a csv file, but I haven't even gotten past testing out file reading yet.
I was having trouble reading the excel files, and kept getting the "System cannot find file specified" error. So I thought I'd see if I could at least get it to read a simple text file, using an example from golangbot, which looks like this:
package main
import (
"fmt"
"io/ioutil"
)
func main() {
data, err := ioutil.ReadFile("test.txt")
if err != nil {
fmt.Println("File reading error", err)
return
}
fmt.Println("Contents of file:", string(data))
}
That simple. The text file is located within the same folder (in %USERPROFILE%/go/src, and /go/ is my GOPATH) as the actual code file I'm attempting to run, and yes, it is called "test.txt". Yet, every attempt to run gives me the same error message, that the system cannot find the specified file (test.txt).
Running any other kind of .go file or building one from this location works just fine. I'm seen others with this error, but it seemed like it was always to do with the GOPATH being set wrong.
I'm frustrated that I even have to ask about something like this, but it's all I could think of right now. Is there something wrong with the locations of my files or the GOPATH itself, or is this something different?
Thank you
Welp, the problem was solved. Turns out, the actual name of the txt file was test.txt.txt. Thanks to notepad and my own lack of awareness.
Bit embarrassing, really. Changing the name worked.
when you are trying this with your notepad. Be cautious while saving it.
The file type should be compatible
File -> save as >
FileName : FileName.go
Save as type : All Files
Now execute as ("folder path">go run FileName.go
Then the result. Kudos
I have been having a similar issue as well. I would try to do this in order to fix it. delete the file and create a new one. That is a simple solution in my opinion. Make sure to copy the code and then paste onto the new file. Mkdir
cd into that directory touch or nano
create a new file. then open that file. then do go run that file name.
it should work.
Oracle's sqlldr defaults to a .dat extension. That I want to override. I don't like to rename the file. When googled get to know few answers to use . like data='fileName.' which is not working. Share your ideas, please.
Error message is fileName.dat is not found.
Sqlloder has default extension for all input files data,log,control...
data= .dat
log= .log
control = .ctl
bad =.bad
PARFILE = .par
But you have to pass filename without apostrophe and dot
sqlloder pass/user#db control=control data=data
sqloader will add extension. control.ctl data.dat
Nevertheless i do not understand why you do not want to specify extension?
You can't, at least in Unix/Linux environments. In Windows you can use the trailing period trick, specifying either INFILE 'filename.' in the control file or DATA=filename. on the command line. WIndows file name handling allows that; you can for instance do DIR filename. at a command prompt and it will list the file with no extension (as will DIR filename). But you can't do that with *nix, from a shell prompt or anywhere else.
You said you don't want to copy or rename the file. Temporarily renaming it might be the simplest solution, but as you may have a reason not to do that even briefly you could instead create a hard or soft link to the file which does have an extension, and use that link as the target instead. You could wrap that in a shell script that takes the file name argument:
# set variable from correct positional parameter; if you pass in the control
# file name or other options, this might not be $1 so adjust as needed
# if the tmeproary file won't be int he same directory, need to be full path
filename=$1
# optionally check file exists, is readable, etc. but overkill for demo
# can also check temporary file does not already exist - stop or remove
# create soft link somewhere it won't impact any other processes
ln -s ${filename} /tmp/${filename##*/}.dat
# run SQL*Loader with soft link as target
sqlldr user/password#db control=file.ctl data=/tmp/${filename##*/}.dat
# clean up
rm -f /tmp/${filename##*/}.dat
You can then call that as:
./scriptfile.sh /path/to/filename
If you can create the link in the same directory then you only need to pass the file, but if it's somewhere else - which may be necessary depending on why renaming isn't an option, and desirable either way - then you need to pass the full path of the data file so the link works. (If the temporary file will be int he same filesystem you could use a hard link, and you wouldn't have to pass the full path then either, but it's still cleaner to do so).
As you haven't shown your current command line options you may have to adjust that to take into account anything else you currently specify there rather than in the control file, particularly which positional argument is actually the data file path.
I have the same issue. I get a monthly download of reference data used in medical application and the 485 downloaded files don't have file extensions (#2gb). Unless I can load without file extensions I have to copy the files with .dat and load from there.
When I open file in mode like this:
file, _ := os.OpenFile("/path/to/my/file", os.O_RDWR|os.O_APPEND, os.FileMode(0666))
file.Seek(start, os.SEEK_SET)
io.CopyN(file, resp.Body, length)
io.CopyN does not respect the position where I sought. It seems it just append to the tail of the file. Instead if I open the file like this:
file, _ := os.OpenFile("/path/to/my/file", os.O_RDWR, os.FileMode(0666))
file.Seek(start, os.SEEK_SET)
io.CopyN(file, resp.Body, length)
It works as I expected. io.CopyN writes to the file from the "start" point I sought. Not sure if this is a feature or a bug?
It's definitely a feature (http://man7.org/linux/man-pages/man2/open.2.html) and it's controlled by underlying OS, not golang runtime.
O_APPEND
The file is opened in append mode. Before each write(2), the
file offset is positioned at the end of the file, as if with
lseek(2). O_APPEND may lead to corrupted files on NFS
filesystems if more than one process appends data to a file at
once. This is because NFS does not support appending to a
file, so the client kernel has to simulate it, which can't be
done without a race condition.
I am using twitter ruby gem to fetch twitter search result. The example code from Github extracts the information from search result.I am wondering how to save the search result, which is JSON i think, to a separate JSON file.
Here is part of the example code:
results = #search.perform("$aaa", 1000)
aFile = File.new("data.txt", "w")
results.map do |status|
myStr="#{status.from_user}: #{status.text} #{status.created_at}"
aFile.write(myStr)
aFile.write("\n")
end
Is there any way to save all the search result to a separate JSON file instead of writing strings to a file?
Thanks in advance.
If you want to save to a file all you need to do is open the file, write it it, then close it:
File.open("myFileName.txt", "a") do |mFile|
mFile.syswrite("Your content here")
mFile.close
end
When you use open you will create the file if it doesn't exist.
One thing to be aware of is that there are different ways to open file, of which will determine where the program writes to. The "a" indicates that it will append everything you write to the file, to the end of the current content.
Here is some of the options:
r Read-only mode. The file pointer is placed at the beginning of the file. This is the default mode.
r+ Read-write mode. The file pointer will be at the beginning of the file.
w Write-only mode. Overwrites the file if the file exists. If the file does not exist, creates a new file for writing.
w+ Read-write mode. Overwrites the existing file if the file exists. If the file does not exist, creates a new file for reading and writing.
a Write-only mode. The file pointer is at the end of the file if the file exists. That is, the file is in the append mode. If the file does not exist, it creates a new file for writing.
a+ Read and write mode. The file pointer is at the end of the file if the file exists. The file opens in the append mode. If the file does not exist, it creates a new file for reading and writing.
So in your case, you would want to pull out the data you want to save, then write it to a file as I have shown. You can also specify file paths by doing:
File.open("/the/path/to/yourfile/myFileName.txt", "a") do |mFile|
mFile.syswrite("Your content here")
mFile.close
end
Another thing to be aware of is that open does not create directories, so you will either need to create directories yourself, or you can do it with your program. Here is a link that is helpful for file input/output:
http://www.tutorialspoint.com/ruby/ruby_input_output.htm
When the code below is ran, the watch is only triggered if I edit and save tmp.txt manually, using either my ide, TextEditor.app, or vim.
It doesn't by method of the write stream or manual shell output redirection (typing echo "test" > /path/to/tmp.txt").
Although if I watch the file itself, and not its dirname, then it works.
var fs, Path, file, watchPath, w;
fs = require('fs' );
Path = require('path');
file = __dirname + '/tmp.txt';
watchPath = Path.dirname(file); // changing this to just file makes it trigger
w = fs.watch ( watchPath, function (e,f) {
console.log("will not get here by itself");
w.close();
});
fs.writeFileSync(file,"test","utf-8");
fs.createWriteStream(file, {
flags:'w',
mode: 0777
} )
.end('the_date="'+new Date+'";' ); // another method fails as well
setTimeout (function () {
fs.writeFileSync(file,"test","utf-8");
},500); // as does this one
// child_process exec and spawn fail the same way with or without timeout
So the questions are: why? and how to trigger this event programmatically from a node script?
Thanks!
It doesn't trigger because a change to the contents of a file isn't a change to the directory.
Under the covers, at least as of 0.6, fs.watch on Mac uses kqueue, and it's a pretty thin wrapper around kqueue file system notifications. So, if you really want to understand the details, you have to understand kqueue, and inodes and things like that.
But if you want a short "lie-to-children" explanation: What a user thinks of as a "file" is really two separate things—the actual file, and the directory entry that points to the actual file. This is what allows you to have things like hard links, and files that can still be read and written even after you've deleted them, and so on.
In general, when you write to an existing file, this doesn't make any change to the directory entry, so anyone watching the directory won't see any change. That's why echo >tmp.txt doesn't trigger you.
However, if you, e.g., write a new temporary file and then move it over the old file, that does change the directory entry (making it a pointer to the new file instead of the old one), so you will be notified. That's why TextEditor.app does trigger you.
The thing is, you've asked to watch the directory and not the file.
The directory isn't updated when the file is modified, such as via shell redirection; in this case, the file is opened, modified, and closed. The directory isn't changed -- only the file is.
When you use a text editor to modify a file, the usual set of system calls behind the scenes looks something like this:
fd = open("foo.new")
write(fd, new foo contents)
unlink("foo")
rename("foo.new", "foo")
This way, the foo file is either entirely the old file or entirely the new file, and there's no way for there to be a "partial file" with the new contents. The renaming operations do modify the directory, thus triggering the directory watch.
Although the above answers seems reasonable, they are not fully accurate. It is actually a very useful feature to be able to listen to a directory for file changes, not just "renames". I think this feature works as expected in Windows at least, and in node 0.9.2 is also working for mac since they changed to the FSEvents API that supports the feature:
Version 0.9.2 (Unstable)