I need to extract numbers from several files into a cumulative variable and print that variable for each parent directory as shown
├───Parent1
│ ├───20210824_2000
│ │ ├───200000_child1
│ │ │ report.md
│ │ │ log.log
│ │ │ file.xml
│ │ │ input.json
│ │ │
│ │ ├───200030_child2
│ │ │
│ │ ├───200034_child3
│ │ │
│ │ ├───200039_child4
│ │ ...
│ ├───20210825_0800
│ │ ├───200000_child1
│ │ │ report.md
│ │ │ log.log
│ │ │ file.xml
│ │ │ input.json
│ │ │
│ │ ├───200030_child2
│ │ │
│ │ ├───200034_child3
│ │ │
│ │ ├───200039_child4
│ │ ...
│ ...
├───Parent2
│ ├───20210824_2000
│ │ ├───200000_child1
│ │ │ report.md
│ │ │ log.log
│ │ │ file.xml
│ │ │ input.json
│ │ │
│ │ ├───200030_child2
│ │ │
│ │ ├───200034_child3
│ │ │
│ │ ├───200039_child4
│ │ ...
│ ├───20210825_0800
│ │ ├───200000_child1
│ │ │ report.md
│ │ │ log.log
│ │ │ file.xml
│ │ │ input.json
│ │ │
│ │ ├───200030_child2
│ │ │
│ │ ├───200034_child3
│ │ │
│ │ ├───200039_child4
│ │ ...
│ ...
...
I can't seem to extract the grep output into a numeric variable. The child folder has a timestamp attached so I sort since I only want the latest files.
Here's what I have so far:
#!/bin/bash
find . -type d -iname 'parent*' | while read -r dir; do
sum=0;
find "$dir" -maxdepth 1 -type d | sort -r | head -1 |
(
while read -r subdir; do
count="$(find "$subdir" -type f -iname '*report.md' -exec grep -ohP '(?<=\*)\d+(?=\*+ number of things)' {} \+)"
sum=$((sum + count))
done
basename "$dir" "$sum"
)
done
But this doesn't seem to want to add count to sum it rather just prints count to the console for each file.
There is a problem due to the sub shell that is created by each while, so variable are not global, you can find useful information in "Shell variables set inside while loop not visible outside of it"
Also count variable could have problem because find should returns multiple line.
Try with this:
#!/bin/bash
find . -type d -iname 'parent*' | while read -r dir; do
sum=0;
while read -r subdir; do
while read report; do
count="$(grep -ohP '(?<=\*)\d+(?=\*+ number of things)' $report)"
sum=$((sum + count))
done < <(find "$subdir" -type f -iname '*report.md')
done < <(find "$dir" -maxdepth 1 -type d | sort -r | head -1)
folder=$(basename $dir)
echo "$folder $sum"
done
Pay attention to the spaces in done < <(.
I have not tested, sorry.
I've using ClickHouse and I have this table
| URL | visits |
| ------------- |:------------- |
| URL1 | 5 |
| URL2 | 30 |
| URL3 | 1 |
| URL4 | 30 |
| URL5 | 9 |
| URL1 | 5 |
| URL2 | 20 |
I can group by url,
select
url,
sum(visits) as visits
from
database.tableVistis
group by
url
| URL | visits |
| ------------- |:------------- |
| URL1 | 10 |
| URL2 | 50 |
| URL3 | 1 |
| URL4 | 30 |
| URL5 | 9 |
And I want this result (group by url, % over total of visits, and % sum accumulate)
| URL | visits | % | Accumulate |
| ------------- |:------------- |----------|------------|
| URL2 | 50 |50% | 50% |
| URL4 | 30 |30% | 80% |
| URL1 | 10 |10% | 90% |
| URL5 | 9 |9% | 99% |
| URL3 | 1 |1% | 100% |
Any idea?
Thanks!!
Try this query:
SELECT result.1 AS URL, result.2 AS visits, round(result.3,2) AS "%", round(result.4, 2) AS Accumulate
FROM (
SELECT
groupArray((URL, visits)) url_visits,
arraySum(x -> x.2, url_visits) total_visits,
arrayMap(x -> (100 / total_visits) * x.2, url_visits) percent_visits,
arrayCumSum(percent_visits) acc_percent_visits,
arrayJoin(arrayMap((x, y, z) -> (x.1, x.2, y, z), url_visits, percent_visits, acc_percent_visits)) result
FROM (
SELECT URL, sum(visits) visits
FROM (
/* test data */
SELECT data.1 URL, data.2 visits
FROM (
SELECT arrayJoin([
('URL1', 5 ),
('URL2', 30),
('URL3', 1 ),
('URL4', 30),
('URL5', 9 ),
('URL1', 5 ),
('URL2', 20)]) data))
GROUP BY URL
ORDER BY visits DESC))
/* result
┌─URL──┬─visits─┬──%─┬─Accumulate─┐
│ URL2 │ 50 │ 50 │ 50 │
│ URL4 │ 30 │ 30 │ 80 │
│ URL1 │ 10 │ 10 │ 90 │
│ URL5 │ 9 │ 9 │ 99 │
│ URL3 │ 1 │ 1 │ 100 │
└──────┴────────┴────┴────────────┘
*/
Yea! Thanks!!
SELECT
result.1 AS URL,
result.2 AS visits,
round(result.3,
2) AS "%",
round(result.4,
2) AS Accumulate
FROM
(
SELECT
groupArray((URL,
visits)) url_visits,
arraySum(x -> x.2,
url_visits) total_visits,
arrayMap(x -> (100 / total_visits) * x.2,
url_visits) percent_visits,
arrayCumSum(percent_visits) acc_percent_visits,
arrayJoin(arrayMap((x,
y,
z) -> (x.1,
x.2,
y,
z),
url_visits,
percent_visits,
acc_percent_visits)) result
FROM
(
SELECT
URL,
sum(visits) visits
FROM
(/* test data */
SELECT
URL,
visits
FROM
(
select
landing as URL,
visitas as visits
from
Analytics
where
fecha > '2019-05-01'))
GROUP BY
URL
ORDER BY
visits DESC))
This query is more compact:
SELECT
result.1 AS URL,
result.2 AS visits,
round(result.3, 2) AS `%`,
round(result.4, 2) AS Accumulate
FROM
(
SELECT
groupArray((URL, visits)) AS url_visits,
arraySum(x -> (x.2), url_visits) AS total_visits,
arrayMap(x -> ((100 / total_visits) * (x.2)), url_visits) AS percent_visits,
arrayCumSum(percent_visits) AS acc_percent_visits,
arrayJoin(arrayMap((x, y, z) -> (x.1, x.2, y, z), url_visits, percent_visits, acc_percent_visits)) AS result
FROM
(
SELECT
landing AS URL,
sum(visitas) AS visits
FROM Analytics
WHERE fecha > '2019-05-01'
GROUP BY URL
ORDER BY visits DESC
)
)
I have got a table with purchases:
player_id | date | registration_date | price
pl1 | 2019-01-21 | 2019-01-20 | 20
pl1 | 2019-01-23 | 2019-01-20 | 10
pl1 | 2019-01-24 | 2019-01-20 | 15
After calculations with groupArray on 'date' and arrayCumSum on 'price' and using ArrayJoin I got the table with cumulative sum on each day:
player_id | date | registration_date | sum_price
pl1 | 2019-01-21 | 2019-01-20 | 20
pl1 | 2019-01-23 | 2019-01-20 | 30
pl1 | 2019-01-24 | 2019-01-20 | 45
But I need to add missing dates since registration until today (today is '2019-01-25'):
player_id | date | registration_date | sum_price
pl1 | 2019-01-20 | 2019-01-20 | 0
pl1 | 2019-01-21 | 2019-01-20 | 20
pl1 | 2019-01-22 | 2019-01-20 | 20
pl1 | 2019-01-23 | 2019-01-20 | 30
pl1 | 2019-01-24 | 2019-01-20 | 45
pl1 | 2019-01-25 | 2019-01-20 | 45
How can I do it?
Try this one:
SELECT player_id, result.1 as date, registrationDate as registration_date, result.2 as sum_price
FROM
(
SELECT
player_id,
groupArray((date, price)) AS purchases,
min(registration_date) AS registrationDate,
arrayMap(x -> registrationDate + x, range(toUInt32(toDate('2019-01-25') - registrationDate + 1))) dates,
arrayFilter(x -> arrayFirstIndex(p -> p.1 = x, purchases) = 0, dates) AS missed_dates,
arrayMap(x -> (x, 0), missed_dates) AS dummy_purchases,
arraySort(x -> x.1, arrayConcat(purchases, dummy_purchases)) all_purchases,
arrayCumSum(x -> x.2, all_purchases) cum_prices,
arrayMap(index -> (all_purchases[index].1, cum_prices[index]), arrayEnumerate(all_purchases)) flat_result,
arrayJoin(flat_result) result
FROM test.purchases01
GROUP BY player_id
)
/* result
┌─player_id─┬───────date─┬─registration_date─┬─sum_price─┐
│ pl1 │ 2019-01-20 │ 2019-01-20 │ 0 │
│ pl1 │ 2019-01-21 │ 2019-01-20 │ 20 │
│ pl1 │ 2019-01-22 │ 2019-01-20 │ 20 │
│ pl1 │ 2019-01-23 │ 2019-01-20 │ 30 │
│ pl1 │ 2019-01-24 │ 2019-01-20 │ 45 │
│ pl1 │ 2019-01-25 │ 2019-01-20 │ 45 │
└───────────┴────────────┴───────────────────┴───────────┘
*/
/* Prepare test data */
CREATE TABLE test.purchases01
(
`player_id` String,
`date` Date,
`registration_date` Date,
`price` int
)
ENGINE = Memory;
INSERT INTO test.purchases01
VALUES ('pl1', '2019-01-21', '2019-01-20', 20),
('pl1', '2019-01-23', '2019-01-20', 10),
('pl1', '2019-01-24', '2019-01-20', 15);
I'm not the first to have these symptoms.. but none of the causes I found so far seem to be the same as I am experiencing.
After I deployed my Laravel app some of the assets return a 404 status. I have checked the permissions for these files.
Other files in the same folder also loaded with the asset() method load just fine.
I have set up my app in a folder at the same level as public_html and copied the contents of public/ to my public_html/subdomain folder.
The issue can be seen here: Staging area
This is my file structure for public_html/subdomain/:
|-- cgi-bin
|-- css
| |-- account.css
| |-- admin_custom.css
| |-- app.css
| |-- home.css
| `-- nav.css
|-- favicon.ico
|-- images
| |-- venray-bloeit-logo-black.png
| |-- venray-bloeit-logo-white.png
| `-- wave_footer.png
|-- index.php
|-- js
| |-- account.js
| |-- admin
| | |-- accounts.js
| | |-- admin.js
| | `-- dashboard.js
| |-- app.js
| |-- nav.js
| `-- plugins
| |-- backstretch.min.js
| |-- jquery-3.2.1.min.js
| `-- tinymce
| |-- jquery.tinymce.min.js
|
|-- robots.txt
|-- vendor
| |-- adminlte
| | |-- css
| | | `-- auth.css
| | |-- dist
| | | |-- css
| | | | |-- AdminLTE.css
| | | | |-- AdminLTE.min.css
| | | | `-- skins
| | | | |-- skin-black-light.css
| | | `-- js
| | | |-- adminlte.js
| | | `-- adminlte.min.js
| | |-- plugins
| | | `-- iCheck
| | | |-- icheck.js
| | | `-- icheck.min.js
| | `-- vendor
| | `-- jquery.min.map
| `-- notify
| |-- danger.mp3
| |-- info.mp3
| |-- notify.css
| `-- notify.js
`-- web.config
How do I explicitly tell shadowJar to use a Kotlin file as the entry point for the resulting uberJar? (Or, fat JAR as you prefer.)
clumsily editing META-INF/MANIFEST.MF and the replacing it in the zip file (used the GUI to put it back):
thufir#dur:~/NetBeansProjects/kotlin/build/libs$
thufir#dur:~/NetBeansProjects/kotlin/build/libs$ ll
total 900
drwxrwxr-x 2 thufir thufir 4096 Nov 5 03:42 ./
drwxrwxr-x 8 thufir thufir 4096 Nov 5 03:42 ../
-rw-rw-r-- 1 thufir thufir 903184 Nov 5 03:42 kotlinAdder.jar
thufir#dur:~/NetBeansProjects/kotlin/build/libs$
thufir#dur:~/NetBeansProjects/kotlin/build/libs$ jar xf kotlinAdder.jar
thufir#dur:~/NetBeansProjects/kotlin/build/libs$
thufir#dur:~/NetBeansProjects/kotlin/build/libs$ ll
total 928
drwxrwxr-x 7 thufir thufir 4096 Nov 5 03:42 ./
drwxrwxr-x 8 thufir thufir 4096 Nov 5 03:42 ../
drwxrwxr-x 2 thufir thufir 4096 Nov 5 03:42 demo/
drwxrwxr-x 17 thufir thufir 12288 Nov 5 03:42 kotlin/
-rw-rw-r-- 1 thufir thufir 903184 Nov 5 03:42 kotlinAdder.jar
drwxrwxr-x 3 thufir thufir 4096 Nov 5 03:42 META-INF/
drwxrwxr-x 3 thufir thufir 4096 Nov 5 03:42 net/
drwxrwxr-x 4 thufir thufir 4096 Nov 5 03:42 org/
thufir#dur:~/NetBeansProjects/kotlin/build/libs$
thufir#dur:~/NetBeansProjects/kotlin/build/libs$ nano META-INF/MANIFEST.MF
thufir#dur:~/NetBeansProjects/kotlin/build/libs$
thufir#dur:~/NetBeansProjects/kotlin/build/libs$ cat META-INF/MANIFEST.MF
Manifest-Version: 1.0
Main-Class: net.bounceme.dur.kotlin.App
thufir#dur:~/NetBeansProjects/kotlin/build/libs$
thufir#dur:~/NetBeansProjects/kotlin/build/libs$ nano META-INF/MANIFEST.MF
thufir#dur:~/NetBeansProjects/kotlin/build/libs$
thufir#dur:~/NetBeansProjects/kotlin/build/libs$ cat META-INF/MANIFEST.MF
Manifest-Version: 1.0
Main-Class: demo.MainKt
thufir#dur:~/NetBeansProjects/kotlin/build/libs$
thufir#dur:~/NetBeansProjects/kotlin/build/libs$ java -jar kotlinAdder.jar
Hello, world!
thufir#dur:~/NetBeansProjects/kotlin/build/libs$
Success of sorts. It ran the Kotlin class as intended.
project:
.
├── build.gradle
├── gradle
│ └── wrapper
│ ├── gradle-wrapper.jar
│ └── gradle-wrapper.properties
├── gradlew
├── gradlew.bat
├── settings.gradle
└── src
├── main
│ ├── java
│ │ └── net
│ │ └── bounceme
│ │ └── dur
│ │ └── kotlin
│ │ └── App.java
│ └── kotlin
│ └── demo
│ ├── example.kt
│ └── main.kt
└── test
└── java
13 directories, 9 files
build file:
plugins {
id 'com.gradle.build-scan' version '1.8'
// id 'java'
id 'application'
id "org.jetbrains.kotlin.jvm" version "1.1.51"
id 'com.github.johnrengelman.shadow' version '2.0.1'
}
buildScan {
licenseAgreementUrl = 'https://gradle.com/terms-of-service'
licenseAgree = 'yes'
//publishAlways()
}
configurations {
provided
}
shadowJar {
baseName = 'kotlinAdder'
classifier = null
version = null
}
sourceCompatibility = 1.8
targetCompatibility = 1.8
//mainClassName = 'net.bounceme.dur.kotlin.App'
mainClassName = 'demo.MainKt'
repositories {
jcenter()
}
configurations {
provided
}
dependencies {
compile "org.jetbrains.kotlin:kotlin-stdlib:1.1.51"
compile 'com.google.firebase:firebase-auth:11.4.2'
}
How do I tell gradle to keep using shadowJar but to build the MANIFEST.MF as manually edited, to set Main-Class: demo.MainKt explicitly as the entry point.
Else, the Java code runs by default even when I've explicitly set the entry point as above in the build.gradle file.