I got this script to get into a folder and get the key for each file.
function listFilesInFolder(id) {
var folder = DriveApp.getFolderById('');
var contents = folder.getFiles();
var file;
var name;
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName("Imported");
var date;
var size;
sheet.clear();
sheet.appendRow(["Name", "Data", "Size", "Id"]);
while(contents.hasNext()) {
file = contents.next();
name = file.getName();
date = file.getDateCreated()
size = file.getSize()
id = file.getId()
data = [name, date, size, id]
sheet.appendRow(data);
//appendRow
}
};
my problem is that taking too much time to finish going through the whole folder, it usually hits run-time limit before it finishes.
The folder contains 1000+ different files, we automatically upload files daily.
is there any way to make this script more efficient?
I believe your goal as follows.
You want to retrieve the file list just under the specific folder using Google Apps Script.
You want to reduce the process cost of your current script in your question.
Modification points:
In your script, appendRow is used in a loop. In this case, the process cost will be high. Ref
And, I thought that in your situation, when Drive API is used, the process cost might be able to be a bit reduced.
When above points are reflected to your script, it becomes as follows.
Modified script:
Before you use this script, please enable Drive API at Advanced Google services.
function listFilesInFolder(id) {
var folderId = "###"; // Please set the folder ID. If you want to use "id" for this, you can use var folderId = id;
// 1. Retrieve file list using Drive API.
var ar = [["Name", "Data", "Size", "Id"]];
var pageToken = "";
do {
const res = Drive.Files.list({corpora: "allDrives", includeTeamDriveItems: true, supportsAllDrives: true, maxResults: 1000, pageToken: pageToken, q: `'${folderId}' in parents`});
ar = ar.concat(res.items.map(({title, createdDate, fileSize, id}) => [title, createdDate, fileSize || 0, id]));
pageToken = res.nextPageToken;
} while(pageToken);
// 2. Put the file list to Spreadsheet.
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName("Imported");
sheet.clear();
sheet.getRange(1, 1, ar.length, ar[0].length).setValues(ar);
}
Note:
From your question, I couldn't understand about the detail of your situation. So in this modified script, the folders in both your Google Drive and the shared Drive can be searched.
References:
Benchmark: Reading and Writing Spreadsheet using Google Apps Script
Files: list
Related
I am using angularjs and javascript and want to export two arrays to Excel using alasql. The Excel file has two sheets, on every sheet there is one array.
In my Excel result I find an extra column $$hashkey.
According to the information I found, using angularjs, the $$hashkey is automatically removed. I also tried adding 'alasql.options.angularjs' but it did not help.
What am I doing wrong?
I am using the two arrays like this:
$scope.ExecutionsLC1: [[Execution,1,2,3],[Operators,1014,1019,1020],[Result,X,X,V]];
$scope.ExecutionsLC2: [[Execution,1,2,3],[Operators,2014,2019,2020],[Result,X,X,V]];
var opts = [{sheetid:'LC1',header:false},{sheetid:'LC2',header:false}];
var res = alasql('SELECT INTO XLSX("LCDetail.xlsx",?) FROM ?',[opts,[$scope.ExecutionsLC1,$scope.ExecutionsLC2]]);
it seems I can use angular.copy() to remove the $$hashkey.
var data1 = angular.copy($scope.ExecutionsLC1);
var data2 = angular.copy($scope.ExecutionsLC2);
var opts = [{sheetid:'One',header:false},{sheetid:'Two',header:false}];
var res = alasql('SELECT INTO XLSX("restest344b.xlsx",?) FROM ?',
[opts,[data1,data2]]);
I am trying to implement caching. I've written code in bounded script of spreadsheet. It's working fine i.e. I am able to get values against particular key. But this code is valid only for bounded script.
Does anyone know that how to access value against particular key from separate script?
Code to put in cache:(Bounded Script)
var sp_key = '1231232';
var ss = SpreadsheetApp.openById(sp_Key);
var s = ss.getSheetByName("test_Sheet");
var cache = CacheService.getScriptCache();
var val= "xyz"
cache.put(A, val);
var cache = CacheService.getPublicCache();
Logger.log(cache.get(A));
Above code works fine. But if I want to get the value from unbounded script then what is the best way?
The getScriptCache() method also works in a stand alone Apps Script project.
There is an error in your code. A is not defined. Either put quotes around A or define A as a variable, and assign a value
function scriptCache() {
var sp_key = '1231232';
//var ss = SpreadsheetApp.openById(sp_Key);
//var s = ss.getSheetByName("test_Sheet");
var cache = CacheService.getScriptCache();
var val= "xyz"
cache.put('A', val);
var cache = CacheService.getPublicCache();
Logger.log(cache.get('A'));
}
I ran that code in a stand alone Apps Script and it works.
I'm trying to store the ClientID in a local variable but can't work out how to do this ?
To get the ClientId I have been using :
ga(function (tracker) {
clientId = tracker.get('clientId');
});
But I cannot seem to get this function to do anything useful.
What I am looking to do is get this ClientID into a local variable using VBScript to then store this and append it to various items.
Usually this would be:
strLocalClientID= whateverFunction()
Thanks,
Rick
One way would be to collect all tracker objects (unless you know the trackers name) and use the get method just like in the example
var trackers = ga.getAll();
for (var i=0; i < trackers.length; ++i) {
var tracker = trackers[i];
console.log(tracker.get('clientId'));
}
If you have just one tracker (which will be usually the case) you can simplify this to
var trackers = ga.getAll();
var clientId = trackers[0].get('clientId');
I'm using the LinqToExcel library. Working great so far, except that I need to start the query at a specific row. This is because the excel spreadsheet from the client uses some images and "header" information at the top of the excel file before the data actually starts.
The data itself will be simple to read and is fairly generic, I just need to know how to tell the ExcelQueryFactory to start at a specific row.
I am aware of the WorksheetRange<Company>("B3", "G10") option, but I don't want to specify an ending row, just where to start reading the file.
Using the latest v. of LinqToExcel with C#
I just tried this code and it seemed to work just fine:
var book = new LinqToExcel.ExcelQueryFactory(#"E:\Temporary\Book1.xlsx");
var query =
from row in book.WorksheetRange("A4", "B16384")
select new
{
Name = row["Name"].Cast<string>(),
Age = row["Age"].Cast<int>(),
};
I only got back the rows with data.
I suppose that you already solved this, but maybe for others - looks like you can use
var excel = new ExcelQueryFactory(path);
var allRows = excel.WorksheetNoHeader();
//start from 3rd row (zero-based indexing), length = allRows.Count() or computed range of rows you want
for (int i = 2; i < length; i++)
{
RowNoHeader row = allRows.ElementAtOrDefault(i);
//process the row - access columns as you want - also zero-based indexing
}
Not as simple as specifying some Range("B3", ...), but also the way.
Hope this helps at least somebody ;)
I had tried this, works fine for my scenario.
//get the sheets info
var faceWrksheet = excel.Worksheet(facemechSheetName);
// get the total rows count.
int _faceMechRows = faceWrksheet.Count();
// append with End Range.
var faceMechResult = excel.WorksheetRange<ExcelFaceMech>("A5", "AS" + _faceMechRows.ToString(), SheetName).
Where(i => i.WorkOrder != null).Select(x => x).ToList();
Have you tried WorksheetRange<Company>("B3", "G")
Unforunatly, at this moment and iteration in the LinqToExcel framework, there does not appear to be any way to do this.
To get around this we are requiring the client to have the data to be uploaded in it's own "sheet" within the excel document. The header row at the first row and the data under it. If they want any "meta data" they will need to include this in another sheet. Below is an example from the LinqToExcel documentation on how to query off a specific sheet.
var excel = new ExcelQueryFactory("excelFileName");
var oldCompanies = from c in repo.Worksheet<Company>("US Companies") //worksheet name = 'US Companies'
where c.LaunchDate < new DateTime(1900, 0, 0)
select c;
in my current ASP.net MVC 3.0 project i am stuck with a situation.
I have four .txt files each has approximatly 100k rows of records
These files will be replaced with new files on weekly bases.
I need to Query data from these four text files, I am not able to choose the best and efficient way to do this.
3 ways I could think
Convert these text files to XML on a weekly basis and query it with Linq-XML
Run a batch import weekly from txt to SQL Server and query using Linq-Entities
avoid all conversions and query directly from text files.
Can any one suggest a best way to deal with this situation.
update:
Url of the Text File
I should connect to this file with credentials.
once i connect successfully, I will have the text file as below with Pipeline as Deliminator
This is the text file
Now i have to look up for the field highlighted in yellow and get the data in that row.
Note: First two lines of the text file are headers of the File.
Well As i Found a way my self. Hope this will be useful for any who are interested to get this done.
string url = "https://myurl.com/folder/file.txt";
WebClient request = new WebClient();
request.Credentials = new NetworkCredential(ConfigurationManager.AppSettings["UserName"], ConfigurationManager.AppSettings["Password"]);
Stream s = request.OpenRead(url);
using (StreamReader strReader = new StreamReader(s))
{
for (int i = 0; i <= 1; i++)
strReader.ReadLine();
while (!strReader.EndOfStream)
{
var CurrentLine = strReader.ReadLine();
var count = CurrentLine.Split('|').Count();
if (count > 3 && CurrentLine.Split('|')[3].Equals("SearchString"))
{
#region Bind Data to Model
//var Line = CurrentLine.Split('|');
//CID.RecordType = Line[0];
//CID.ChangeIdentifier = Line[1];
//CID.CoverageID = Convert.ToInt32(Line[2]);
//CID.NationalDrugCode = Line[3];
//CID.DrugQualifier = Convert.ToInt32(Line[4]);
#endregion
break;
}
}
s.Close();
}
request.Dispose();