Now I want to download the dataset on the website(http://pubchemqc.riken.jp/), it's in google drive and filesize is 2TB. the website recommend using rclone to download the file but not said how to use.
As the picture shows
This was asked a while back already but I'd like to keep it recorded here as it seems hard to find.
Unfortunately I don't know if there's a way to automate this process but it's still easy.
What you can do is create a new remote with the link's root_folder_id.
The Root Folder ID is present in the url e.g:
in the example link
https://drive.google.com/drive/folders/a1b2c3deFgHi4JKlm56nOpqrStuv7w8xy9
the root_folder_id is the string after the slash /folders/ so in this case it would be "a1b2c3deFgHi4JKlm56nOpqrStuv7w8xy9". If the Google Drive shared link you got is a folder all you have to do is copy that id and use it in the rclone's new remote setup. Now if the shared link points directly to a file, like the one in the OP, there's no folder to download it from, so we gotta create our own! The whole process would look like:
get the folder_id from Google Drive's link. If it is a link to a direct file then first we have to create a new folder anywhere inside our own Google Drive, it's name doesn't matter as we will point directly to it using the ID. After creating this new folder, open it and note the url, it should look something like
https://drive.google.com/drive/u/1/folders/Fdrcv3nQvxQqXUGEEyvacwUxdYXpV33Ct
just copy everything after /folders/ and save it for later.
now go back to the link of the direct file. Add a shortcut for that file to inside your recently created folder, so say the folder was named dl-with-rclone, click the "Add shortcut to Drive" icon and navigate to the "dl-with-rclone" folder to add the shortcut there.
heading to rclone, do:
rclone config <to open settings>
On the prompt e/n/d/r/c/s/q> hit n (the key relative to creating a new remote)
On name> give it a name like sharedWithMe or anything else for personal reference later
On the prompt Storage>, the possible remotes to connect to, write down drive or the number relative to it (currently 15)
if you set up your own client_id and client_secret you put them next
On the scope> prompt option 2 "Read-only" is enough
now this is the important one:
on the root_folder_id> prompt input the ID of the folder of the shared link or the folder you created and pointed the file shortcut to, e.g. Fdrcv3nQvxQqXUGEEyvacwUxdYXpV33Ct
Now you can pretty much hit enter for everything, once you reach Use auto config? you will be redirected for a browser to login. Make sure the logged in account is the same that you created the folder for the shortcut. If the shared link is for a folder already your logged in account doesn't matter.
After finishing your remote setup you can exit rclone config
The command you would need would be something like rclone copy sharedWithMe: destination/folder
This being rclone you would surely be able to copy from one remote to another, that's up to the user.
In summary: I did test this method on your link, creating a folder in my Drive, pointing the file shortcut to it and using the root_folder_id to setup the rclone remote and it did begin to download:
rclone test download screenshot
Related
this is a general question about Power Automate and OneDrive that I've seen no solution to.
I'm trying to create a flow "Copy files from a folder in OneDrive (Business) to an FTP server". The trigger is "When a filed is created" in a OneDrive directory. When I attempt to navigate to the folder I just see ROOT, then "No Items".
I also can't figure out how to obtain the unique identifier of the folder.
I was able to get the ID of my folders with this short Instant flow.
It'll output a JSON response with the ID, Name, Path, etc for all your folders in the Root of OneDrive. There is also a "List files in folder" block that you can use to get subfolders of folders.
I was able to copy the ID string from the JSON response and paste it into the "When a file is create" OneDrive trigger and have it go off successfully.
The issue was I was attempting to do this using OneDrive(Business) when I should have used Sharepoint based on my organization's license
My application downloads a PDF and stores it in the LocalState folder for my Windows 8 app.
I have a link within the app that I would like to show the PDF when the user clicks it.
I've tried displaying it using ms-appdata:///local/pdfs/filename.pdf in a window.open call and I also tried using the InAppBrowser plugin within cordova with no luck. Additionally, I've tried the following:
var uri = new Windows.Foundation.Uri('ms-appdata:///local/pdfs/filename.pdf');
var file = Windows.Storage.StorageFile.getFileFromApplicationUriAsync(uri);
Windows.System.Launcher.launchFileAsync(file).done();
I know the file exists as I'm getting a file result back. Just not sure how to allow the user to view it.
By design, the local appdata folder on Windows is accessible only to that app, or to full-trust desktop applications (and this is probably true of similar sandboxed locations on other platforms). As a result, a Windows Store app that gets launched with Launcher.launchFileAsync won't be able access that location (nor can a webview process, which is also sandboxed). If a desktop application gets launched, on the other hand, it probably can access the file, but you can't tell ahead of time if that's the case. Bottom line is that local appdata isn't a good location for letting other apps get at the file.
You'll need to save the file in another location that is accessible to other apps. There are two approaches here, both of which will require a little user interaction to select a location, so they can place the PDFs anywhere they want:
Have the user select a save folder for your app, which they can do once. You would invoke the FolderPicker for this purpose, and save the selected folder in the FutureAccessList. This way you can have the user select the save folder, which grants you consent to save there, and by saving it in the FutureAccessList you can retrieve it in subsequent sessions without having to ask the user again. Refer to the File Picker Sample and the File Access Sample for more.
Have the user select a save location for each individual file, using the FilePicker (see the same sample), and you can also use the access cache to save permissions to those individual locations if you need them later.
There might be Cordova plugins that work with these APIs too, but I haven't checked. Either way, once the file is in an accessible location, launching the file should work just fine.
As an alternate solution, you could consider rendering the PDFs directly in your app. Windows has an API for this in Windows.Data.Pdf, with an associated sample. There might be a plugin or other JS libraries that could also work for this.
I'm working on an intranet web application with a file sharing server set up using AFP.
My goal is to create a link on the page that will open up Finder to a specific folder on the share network, but it has to go through an alias. I think it'd be easier to explain with an example.
For this share, there is a folder called Objects that contains a bunch of folders with unpredictable names, and a folder called Alias that contains a bunch of aliases to folders in Objects. So a link to afp://server/share/Alias/obj1 should open the folder at afp://server/share/Objects/unpredictableName, where obj1 is an alias pointing to unpredicableName.
On OSX 10.8 this works as expected, however on 10.9 and later that link opens the Finder at the Alias folder, instead of actually going into unpredictableName. What changed, and is there any way to restore the previous behavior? Thanks.
I'm using PGAdmin 1.14.3.
When I try to execute an import command:
COPY grad(country_code, postal_code, place_name, admin_name1, admin_code1, admin_name2, admin_code2, admin_name3, admin_code3, latitude, longitude, accuracy)
FROM 'C:\\Users\\denis\\Desktop\\BP2Project\\USA\\US.txt';
I get a
ERROR: could not open file
"C:\Users\denis\Desktop\BP2Project\USA\US.txt" for reading:
Permission denied SQL state: 42501
I did look up other similar questions and none of them solved my issue.
I logged in as user "postgres" who is the superuser. I don't see why I'm missing permissions. I'm on Windows 7.
The permissions article mentioned in the answer by Houari and Flimzy is a good reference material, but a direct answer (the quick fix I used) is:
Right click the folder containing the data file(s) that permission was denied to and then click Properties.
In the Folder's Properties window, select the Security tab.
Click the Edit button.
In the "Permissions for the folder" window that opened, click the Add... button.
Type Everyone into the "Enter the object names to select" text area box.
Click OK and the window will close.
Verify that the default Read & Execute permissions were set to Allow via the check checkbox in the previous window.
As JLB notes, Write permission is needed if dumping from PostgreSQL, opposed to copying into it.
Click OK and the window will close.
Click the Apply button in the Folder Properties window.
Now you can run the SQL COPY statement that needs to access those files.
Once done, return to the Folder's Properties window.
Click the Edit button.
Select the Everyone entry in the "Group or user names:" field.
Click the Remove button.
Click OK on the remaining open windows.
The permissions have now been returned to what they were.
The user Postgres must have read access on the file from which you are about to copy.
Look at this article to see how to modify files' security access on Windows.
Ok, this is how got COPY command working,to export a table to CSV, step by step.
Pls note that I am using pgAdmin 111.
Create the target folder you want to export a table to. E.g C:\myExports
Set a read/write permission on this folder following the steps below :
Right click the folder containing the data file(s) that permission was denied >to and then click Properties.
In the Folder's Properties window, select the Security tab.
Click the Edit button.
In the "Permissions for the folder" window that opened, click the Add... button.
Type Everyone into the "Enter the object names to select" text area box.
Click OK and the window will close.
Verify that the default Read & Execute permissions were set to Allow via the >check checkbox in the previous window.
Click OK and the window will close.
Click the Apply button in the Folder Properties window.
This is the tricky part, inside myExports folder create a blank CSV file with your desired name.E.g employee.csv
Then run the Copy command like this :
copy employee to 'C:\myExports\employee.csv' delimiter ',' csv;
employee is the table name in this example..
Hope this helps.
If you don't want to give permissions to Everyone, you can add permissions to the account that started the service. In the Control Panel - Administrative Tools - Services, copy the account name in the 'Log On' tab. (On my system the account is called 'Network Service'.) Then share the folder with the CSV-file with this user as shown in the answer above.
To solve this problem you must give permission to the CSV file because that CSV file present in a COPY command are read directly by the server, but not client application. So to make this file accessible to a server we must give full read-write permission so that Postgresql user can read and write on that file.
Reference: article showing step by step procedure.
I just ran into this error and even after adding postgres to permissions on the file folder and the file itself, it still didn't work. So, I put the file in a public folder. On Windows this was the path: "C:\Users\Public\Documents\census.csv." It worked!
Responses to this problem on different threads go something like this
1. "Tell me exactly what command you used"
2. "Make sure you have right permissions"
3. "Just use /copy"
I just tried giving permissions to Everyone on the cvs file I am trying to copy from, and it is still giving me the permission denied error. I think this functionality is broken and has been broken for multiple consecutive releases over multiple consecutive versions of Windows.
for me and I've just spent some long hours on this.
I have a central db residing on a HP box running 14.04 postgresql-9.5 pgAdmin3 postgis-2.2, shares are made through a tweeked Samba share. My clients are using a mixture of windows 10.1, 7, 8.1 and I have one ubuntu 14.04 desktop.
I'm working with large tables updating records and normalising data and have built the routines around SQL copy statements from CSV files which were made from the core COPY public.table_1 TO (the share folder I'd set up in Samba https://www.youtube.com/watch?v=ndAYZ0DJ-U4) '/srv/samba/share/[filename].csv'
I can then update the database once the tables have been amended with COPY table_1 from '/srv/samba/share/test.csv' USING DELIMITERS ',' WITH NULL AS '' CSV HEADER; from any of my clients.
The key as far as I have been able to determine is that the clients doing the updating must be superusers, also everything must tie up in terms of users as there are 4 servers working together here Postgresql, Samba, UNIX and WINS
All of my users are registered on each of the servers with the same username and password homogeneity is the main factor.
I had tried for a long time moving things about and trying various naming conventions but in the end it was http://www.postgresql.org/message-id/CFF47E56EA077241B1FFF390344B5FC10ACB1C0C#webmail.begavalley.nsw.gov.au that sorted me out it was like a big switch clicking in. chown 777 on your shares and group management was an important learning curve but., the hours I've spent on this will reap rewards down the line... Loving Ubuntu loving life and loving the spirit of open source but that just might be sleep deprivation kicking in... IT WORKS
I am trying to execute SQL commands directly from a file in psql 14, and landed the same error.
The reason is that "postgres" user is different from the 'admin' or the main user of the operating-system. So, this main user denies "postgres" to access files from its file system.
Although there is a way to bypass it.
Windows lets any user access the files in 'C:\Users\Public' and Linux distros allow files in '/tmp' folder to do the same.
So, whatever files you are trying to access from postgres' terminal, keep the files in
'C:\Users\Public' for Windows
'/tmp' in Ubuntu
Read from the orginal source of this answer
use \copy command from psql instead with this config:
sudo psql -U postgres -d <your-db> -c "\copy <your-query-or-table> TO '<pat-to-save-file>' WITH (FORMAT CSV)"
I'm trying to figure out how to autoftp with Filezilla. Are there any other programs that do this right off the bat?
I use WinSCP for some of its amazing benefits. I connect to my FTP server with WinSCP and let it monitor my local folder /website/public_html.
Any changes I make in my local folder automatically get uploaded to my FTP server by WinSCP. The feature is called Keep remote directory up to date:
http://winscp.net/eng/docs/task_keep_up_to_date
As for SCSS, I also use it:
I have a script that opens a console window to watch my /website/scss files and compiles them to /website/public_html/styles/. WinSCP sees the CSS file has changed (or is new) and automatically uploads it.
Notice: The only downside is you cannot reach your error_log as that is automatically generated on the server by PHP. To do this you can easily instantiate another WinSCP to connect to your website. Now you have 1 WinSCP monitoring your local folder and 1 WinSCP where you can access your site map.
If by "autoftp" you mean "update the file on the server whenever I save it" and you're working with code, then I'd recommend the wonderful Notepad++
Notepad++ certainly is wonderful for this particular function and it even saves a cache of the whatever has been accessed and modified from the remote server.
The downside however (for me) is that Notepad++ will not auto-upload preprocessed CSS files like .scss or .less. I tried actually opening the resulting .css files in Notepad++ to mimic the act of editing them, but that still puts an extra step in the process. I have to click over to the tabs and manually save them (after the 'your file as changed prompt').
Currently, after I make changes, I click over to FZ and upload the files manually.
This response is late, but hopefully it will be helpful to those looking to get past Filezilla's "do you want to overwrite" prompt every time a file being edited locally is saved. Unfortunately, the Filezilla developers are staunchly opposed to making this behavior optional. It is a problem for many, especially when working with limited desktop space, since every time the prompt appears, one must expose the hidden filezilla window just to click okay.
Anyway, the answer is WinSCP. It is also free and is so similar to filezilla that I needed no learning to use it immediately.
Cheers!
There is an easier way to do that with NetBeans IDE, there is an option to upload files with FTP or SFTP on save, manually or on Run
Right click on your project and click Properties
Click the second category run configuration
Run As: Choose Remote Website (FTP,SFTP)
Project Url: put your website live link eg www.example.com
Upload files: Choose On Save
Remote Connection > Manage > Add > Connection name (put one) > FTP
Now add your ftp details. Make sure your Initial directory path is correct, (your website path) so you will not accidentally replace any files from other website in the file manager
Unfortunately there no option on filezilla can auto agree upload when file has been changed.
Mobaxtern can do that. You can choose "always upload" when file has been changed.
You can edit the file with any editor, whenever you save, it will upload.
I like to use WebDrive ($40) which mounts a remote server (S)FTP/SSH/Cloud to a local drive (eg. X: on Windows). You can then just do everything to drive X: as if it's a local drive and WebDrive automatically uploads the changes to the remote server.