I want to download all the 7000 suggestions of the PDB files of that specific protein in RCSB, How should I proceed?
I know that there is a download option beside each, but is there any shortcut to download all in one set?
If you have a list of all the pdbs that you want to download you can use this function to download the pdb files
from requests import get
import os
def donwload_pdb(pdb_id: str,
download_dir: str = os.path.join(os.path.expanduser('~/'),
'Downloads')) -> None:
r = get(f"https://files.rcsb.org/download/{pdb_id.upper()}.pdb")
open(os.path.join(download_dir, f"{pdb_id.upper()}.pdb"), 'w').write(r.text)
return
pdbids: list[str] = [1xzy, 2xyz, ...]
for pdbid in pdbids:
download_pdb(pdb_id = pdbid, donwload_dir='path/to/dir')
PS: change the download dir to a directory that exists. current default only works on macOS.
Related
I would like to download an entire public folder from Google-drive from script (python, wget, terminal, etc.).
The procedure shall be accomplished without authentication, as it's a public folder which is accessible for anyone who has the link.
Link example: https://drive.google.com/drive/folders/1Gt-W8jMrADizXYGF5QZwJh_Gc8QpKflX
In case that it's not possible to directly download the entire folder, then it would be sufficient to just being able to list its content (files), without authentication, and then I'll be able to download each file separately. How to obtain such a listing feature?
Note:
I found many similar discussions, but all were assuming either file-download or authentication, and I couldn't find a match for that specific ask, e.g.:
Python: How do download entire folder from Google Drive
download folder from google drive
Download entire Google Drive folder from the shared link using Google drive API
How to download specific Google Drive folder using Python?
Download Shared Google Drive Folder with Python
Python: download files from google drive using url
https://unix.stackexchange.com/questions/136371/how-to-download-a-folder-from-google-drive-using-terminal
https://github.com/vikynandha-zz/google-drive-backup/blob/master/drive.py
I've ended up applying the following code (tested, works well):
import urllib.request
from getfilelistpy import getfilelist
from os import path, makedirs, remove, rename
def download_googledrive_folder(remote_folder, local_dir, gdrive_api_key, debug_en):
success = True
if debug_en:
print('[DEBUG] Downloading: %s --> %s' % (remote_folder, local_dir))
else:
try:
resource = {
"api_key": gdrive_api_key,
"id": remote_folder.split('/')[-1].split('?')[0],
"fields": "files(name,id)",
}
res = getfilelist.GetFileList(resource)
print('Found #%d files' % res['totalNumberOfFiles'])
destination = local_dir
if not path.exists(destination):
makedirs(destination)
for file_dict in res['fileList'][0]['files']:
print('Downloading %s' % file_dict['name'])
if gdrive_api_key:
source = "https://www.googleapis.com/drive/v3/files/%s?alt=media&key=%s" % (file_dict['id'], gdrive_api_key)
else:
source = "https://drive.google.com/uc?id=%s&export=download" % file_dict['id'] # only works for small files (<100MB)
destination_file = path.join(destination, file_dict['name'])
urllib.request.urlretrieve(source, destination_file)
except Exception as err:
print(err)
success = False
return success
I could not find a way to achieve my original goal, i.e. downloading public folder from Google-Drive without any credentials/keys, but the above code is a good compromise for me, as it only requires a key and not full credentials.
Note that there are 2 options here --> with or without providing the Google API Key (gdrive_api_key) for the source URL.
From my experience, the option without the API key works well for small files (< ~100MB), while the option with the API key appear to be more robust and works for any size.
I have a lot of troubles following the instructions form the Kivy website, many steps aren't explained like what should I answer to the warning.
WARNING: The output directory "..." and ALL ITS CONTENTS will be REMOVED! Continue? (y/n)
Even if I choose y, the folder isn't removed.
Also should I always add these lines:
from kivy.deps import sdl2, glew
Tree('C:\\Users\\<username>\\Desktop\\MyApp\\'),
*[Tree(p) for p in (sdl2.dep_bins + glew.dep_bins)]
in the .spec file? Why are they necessary?
Not many info is available for Kivy.
Because I spent a lot of time understanding how I should package my app, here are some instructions that would have really helped me.
Some info are available at http://pythonhosted.org/PyInstaller/
Python 3.6 as of march 2017
Because packaging my app gave me the error IndexError: tuple index out of range, I had to install the developement version of PyInstaller:
pip install https://github.com/pyinstaller/pyinstaller/archive/develop.zip
Step 1:
I moved all the files of MyApp in a folder "C:\Users\<username>\Desktop\MyApp": the .py, the .kv and the images and I created an icon.ico.
I created another folder C:\Users\<username>\Desktop\MyPackagedApp. In this folder I press Shift+right click and select open command window here.
Then I pasted this:
python -m PyInstaller --name MyApp --icon "C:\Users\<username>\Desktop\MyApp\icon.ico" "C:\Users\<username>\Desktop\MyApp\myapp.py"
This creates two folders, build and dist, and a .spec file. In dist/MyApp, I can find a .exe. Apparently, if my app is really simple (just one label), the packaged app can works without the Step 2.
Step 2:
The second step involves editing the .spec file. Here is an exemple of mine.
(cf Step 3, for the explanations about my_hidden_modules)
I go back to the cmd, and enter
python -m MyApp myapp.spec
I then got this warning:
WARNING: The output directory "..." and ALL ITS CONTENTS will be REMOVED! Continue? (y/n)
I enter y and then press enter.
Because I choosed y, I was surpised that the folder build was still there and that the dist/MyApp was still containing many files. But this is normal. PyInstaller can output a single file .exe or a single folder which contains all the script’s dependencies and an executable file. But the default output is a single folder with multiple files.
Step 3: adding hidden modules
When I click on the myapp.exe in dist/MyApp, the app crashed. In the log C:\Users\.kivy\logs\ I could find 2 errors: ModuleNotFoundError: No module named 'win32timezone' and SystemError: <class '_frozen_importlib._ModuleLockManager'>.
Because of this I had to edit the .spec file and add these lines:
my_hidden_modules = [
( 'C:\\Users\\<username>\\AppData\\Local\\Programs\\Python\\Python36\\Lib\\site-packages\\win32\\lib\\win32timezone.py', '.' )
]
in a = Analysis I changed datas = [] to datas = my_hidden_modules,
Apparently this is because I used a FileChooser widget.
So, the line:
ALL ITS CONTENTS will be REMOVED!
yes, it will be removed AND replaced later with new files. Check the date. I think it prints permission denied if it can't do such a thin both for files and the whole folder, so you'd notice it. It's important though, because you need to add additional files into your folder.
Those additional files of two types:
kivy dependencies
application data
Dependencies are just binaries (+/- loaders, licenses, or so), you get them through the *[Tree(p) ...] piece of code, which is just a command for "get all files from that folder". Without them Kivy won't even start.
Similarly to that, the second Tree(<app folder>) does the same, but for your own files such as .py files, .kv files, images, music, databases, basically whatever you create.
Obviously if you remove the deps, app won't start and if you remove app data, you'll get some path errors and most likely crash. You don't want any of that :P
It also works if in the 'a = Analysis...' block in the spec file one substitutes
hiddenimports=[]
for
hiddenimports=['win32file', 'win32timezone']
for win32file, win32timezone or for whatever files are missing
What I am trying to do is run a batch script to update .pdb files to point to a different source location. I'm not sure if there is a way to update the absolute paths in the pdb file, but currently I'm looking into inserting a source stream into the pdb file so the debugger can get the source files.
I am inserting a source stream using pdbstr.exe:
pdbstr -w -i:c:\test\pdbstream.txt -p:"C:\test\somelibrary.pdb" -s:srcrv
The pdbstream.txt contents are pretty basic:
SRCSRV: ini ------------------------------------------------
VERSION=1
SRCSRV: variables ------------------------------------------
SRCSRVTRG=%var1%
SRCSRVCMD=cmd echo %var1% >> c:\test\output.txt
SRCSRV: source files ---------------------------------------
C:\somedifferentpath\somelibrary\sources\firstclass.cs
C:\somedifferentpath\somelibrary\sources\secondclass.cs
SRCSRV: end ------------------------------------------------
According to the info on MSDN I added only the required parameters. To my understanding, this should run the command in the SRCSRVCMD variable when the debugger loads the pdb file and searches for the source files. So I would expect some output in c:\test\output.txt.
I have verified that the correct pdb file is loaded for the module, but when i'm debugging in Visual Studio it does not find any source file. The output shows:
"SRCSRV: The module 'C:\test\somelibrary.dll' does not contain source server information."
Is the source stream not valid, or am I simply overlooking a particular step? Or can I change the absolute source location in the pdb file without inserting a datastream?
Edit
To clarify, I am currently using the Visual Studio debugger. The code I am trying to debug is in private libraries I created myself. I want the .pdb files to point to the correct source files after everything (dll, pdb & source files) is moved to a different location on disk.
In Steam, we can configure multiple directories/folders where it will look for applications. It is found in the menu Steam->Settings->Downloads->STEAM LIBRARY FOLDERS dialog.
Where can I find those settings if I'm looking programatically from outside Steam?
I'm more interested in the location for the Windows client.
Found it. On Windows they are stored in C:\Program Files (x86)\Steam\SteamApps\libraryfolders.vdf, and you also have to add Steam's install folder C:\Program Files (x86)\Steam
Here's a sample Python script to extract the information:
import re
with open(r"C:\Program Files (x86)\Steam\SteamApps\libraryfolders.vdf") as f:
folders = [r"C:\Program Files (x86)\Steam"]
lf = f.read()
folders.extend([fn.replace("\\\\", "\\") for fn in
re.findall('^\s*"\d*"\s*"([^"]*)"', lf, re.MULTILINE)])
I found it here:
C:\Program Files (x86)\Steam\config\config.vdf
There's a line in that file:
"BaseInstallFolder_1" "{YourSteamLibraryFolder}"
So I just open it with Notepad then Ctrl+F search 'Base'.
If that line is not in there:
Open Steam. > Sign into you account. > 'Steam'. > 'Settings'.
Then click on 'Downloads'. > 'STEAM LIBRARY FOLDERS'.
Make an empty folder somewhere.
Click on 'ADD LIBRARY FOLDER'.
Browse to the empty folder you made. > Click on 'SELECT'.
If you then look at the config.vdf again; there should be a line like this:
"BaseInstallFolder_1" "{YourNewEmptySteamLibraryFolder}"
If Steam hasn't been installed in the default location in Windows, you can find it in the registry under HKEY_LOCAL_MACHINE, on path SOFTWARE\Wow6432Node\Valve\Steam. Here's how I found it in Kotlin:
private val steamFolder: File = File(WinRegistry.getString(
WinRegistry.HKEY_LOCAL_MACHINE,
"SOFTWARE\\Wow6432Node\\Valve\\Steam",
"InstallPath"))
You can then find the libraryfolders.vdf and config.vdf files, as per the other answers.
In each of those library folders you'll find files called appmanifest_<id>.acf, where <id> is the id of the game (find it in Properties / Updates on Steam). If you're looking for a particular game, that will help to determine which folder the game is in, if there's more than one game install location.
I develop Firefox extension with bundled executable file which should be run on browser startup.
To run process I need get nsIFile or nsILocalFile instance which points to executable file.
I know one solution how to get it using directory service:
var file = Components.classes["#mozilla.org/file/directory_service;1"].getService(Components.interfaces.nsIProperties).get("ProfD", Components.interfaces.nsIFile);
file.append("extensions");
file.append("<extension id>");
file.append("<relative path>");
But this solution has two disadvantages:
It doesn't work in development mode, when instead of installed extension I have only text file with real extension path
I'm not sure that it will work on all Firefox configurations because of hardcoded "extensions" part of the path
So is there any nicer way to run executable file which comes with Firefox extension?
Thanks.
You are making way too many assumptions about the directory structure of the Firefox profile - don't. The Add-on Manager API lets you get the path of a file inside the extension, you should use it:
Components.utils.import("resource://gre/modules/AddonManager.jsm");
AddonManager.getAddonByID("<extension id>", function(addon)
{
var uri = addon.getResourceURI("<relative path>");
var file = uri.QueryInterface(Components.interfaces.nsIFileURL).file;
...
});
A restartless addon's startup function (in the bootstrap.js file) will receive, as its first parameter, the path where the addon is installed. You can then play various tricks to read files inside the .jar file, if any: see https://github.com/protz/GMail-Conversation-View/blob/master/bootstrap.js#L55 as an example.
In a non-restartless case, I must confess I don't have much of an idea :).
I found this thread looking for a way to reference a path to an image hosted in extension's directory from a content script. Here's a solution:
Include your files in web_accessible_resources in the extension's manifest.
"web_accessible_resources": [
"images/*"
]
Absolute paths to these resources contain randomly generated UUID, therefore we're using runtime.getUrl() giving it the path relative to manifest.json. Example:
let myImg = document.createElement('img');
myImg.src = browser.runtime.getURL("images/my-img.png")