Is it possible to get User/Channel object if you know user/channel name with Slack API?
If you try to search user/channel in the web version you can see this (trying to find #tmp-blo gives 2 results: #tmp-block-check and #tmp-improving-review-block):
this search use /search resource:
payload:
I can use this request from the Terminal if I save it as cURL:
curl 'https://edgeapi.slack.com/cache/????/channels/search?fp=2b' \
-H 'content-type: text/plain;charset=UTF-8' \
-H 'cookie: d=????; b=??????; lc=????; shown_ssb_redirect_page=1; shown_download_ssb_modal=1; show_download_ssb_banner=1; no_download_ssb_banner=1; _gcl_au=??????; _cs_c=1; _rdt_uuid=????; __qca=????; OptanonConsent=isGpcEnabled=0&datestamp=Tue+Aug+09+2022+17%????&version=6.22.0&isIABGlobal=false&hosts=&consentId=????????&interactionCount=1&landingPath=NotLandingPage&groups=????&AwaitingReconsent=false; _gid=????; _ga=????; _cs_id=????; _lc2_fpi=????; __adroll_fpc=????; __ar_v4=????; _fbp=????; _ga_????=????; d-s=????; x=????' \
-H 'origin: https://app.slack.com' \
-H 'pragma: no-cache' \
-H 'sec-ch-ua: "Chromium";v="104", " Not A;Brand";v="99", "Google Chrome";v="104"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-ch-ua-platform: "macOS"' \
-H 'sec-fetch-dest: empty' \
-H 'sec-fetch-mode: cors' \
-H 'sec-fetch-site: same-site' \
-H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36' \
--data-raw '{"token":"????","query":"tmp-blo","count":30,"fuzz":1,"uax29_tokenizer":false,"check_membership":true}' \
--compressed
but this request requires some authentication cookies. And I don't know how my application should specify appropriate cookies for this request. So my questions are:
can I specify appropriate authorization cookies for /search ? Or is there any other api that can return user/channel id by name?
For a Context:
We have application that periodically sends data to user/channel. User can create its own mailing by specifying the data to send and recipients. We often face a problem that user incorrectly entered user or channel and we want to develop a feature that will suggest options for autocomplete just like Slack Application.
I am trying to create a DbcpController service from nifi rest api. I am getting the proper response also but when i go to UI, The controller service is not visible.
curl -i -X POST -H 'Content-Type:application/json'
-d '{"revision":{"clientId":"a09f2359-0165-1000-bf28-1dca98f6e259","version":0},
"permissions":{"canRead":true,"canWrite":true},"bulletins":[],
"component":{"name":"Dynamic-Pool","type":"org.apache.nifi.dbcp.DBCPConnectionPool",
"bundle":{"group":"org.apache.nifi","artifact":"nifi-dbcp-service-nar","version":"1.6.0"},
"controllerServiceApis":[{"type":"org.apache.nifi.dbcp.DBCPService",
"bundle":{"group":"org.apache.nifi","artifact":"nifi-standard-services-api-nar","version":"1.6.0"}}],
"comments":"","state":"DISABLED","persistsState":false,"restricted":false,"deprecated":false,
"multipleVersionsAvailable":true,"properties":{"Database Connection URL":"jdbc:oracle:thin:#144.21.86.182:1521/ORCL.607022672.oraclecloud.internal",
"Database Driver Class Name":"oracle.jdbc.driver.OracleDriver",
"database-driver-locations":"/home/siddharth/Downloads/ojdbc7.jar","Database User":"test","Password":"***",
"Max Wait Time":"500 millis","Max Total Connections":"8","Validation-query":null},"descriptors":{ } } }'
http://localhost:9090/nifi-api/controller/controller-services/
Response :
HTTP/1.1 201 Created
Date: Mon, 03 Sep 2018 18:19:28 GMT
X-Frame-Options: SAMEORIGIN
Location: http://localhost:9090/nifi-api/controller-services/a0a93568-0165-1000-0c3b-ff9494571881
Content-Type: application/json
Vary: Accept-Encoding
Vary: User-Agent
Content-Length: 3614
Server: Jetty(9.4.3.v20170317)
If i click the location the details also comes up.
Is there any parameter in the request which i am missing.
I am not sure what "When I go to UI, the controller service is not visible" means.
When you go to the Apache NiFi UI, and click Global Menu > Controller Settings > Controller Services, you do not see any Controller Services listed -- Possibly refresh your browser and clear the cache? I don't think this is your issue
The Controller Service is listed but it is not available to use in a processor -- this is because the CS is being added to the global level, which means it is available for Reporting Tasks to reference, but not processors on the canvas. You'll need to provide a parentGroupId in the ControllerServiceDTO object in the JSON in order for the CS to be available to a component on the canvas (in that parent group). Your URL will look like this: /nifi-api/process-groups/a0f3dfb7-0165-1000-8310-4fb72628adaa/controller-services. I think this is your issue
The Controller Service is available to use in a processor but it is not enabled -- this is because you are setting the state of the CS to DISABLED. You can issue an additional command which enables the CS, or set the state to ENABLED initially
If you right click on the canvas and select Configure, you'll get a dialog for the Process Group you're in (called "NiFi Flow" by default at the root level), and there is a tab for "Controller Services", which will list the CS you have created.
You can always perform these operations in the UI and use your browser's Developer Tools panel to monitor the requests -- NiFi's UI is a consumer of the REST API, so any activity performed there can be done using the API alone.
Examples:
Add a reporting task controller service
curl 'http://localhost:8080/nifi-api/controller/controller-services' -H 'Origin: http://localhost:8080' -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: en-US,en;q=0.9' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36' -H 'Content-Type: application/json' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Referer: http://localhost:8080/nifi/' -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' -H 'DNT: 1' --data-binary '{"revision":{"clientId":"a0f490b8-0165-1000-a521-4d24892f12c0","version":0},"disconnectedNodeAcknowledged":false,"component":{"type":"org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService","bundle":{"group":"org.apache.nifi","artifact":"nifi-aws-nar","version":"1.8.0-SNAPSHOT"}}}' --compressed
Enable a reporting task controller service
curl 'http://localhost:8080/nifi-api/controller-services/a0f4dbe2-0165-1000-8cd2-dd50d7dba48d' -X PUT -H 'Origin: http://localhost:8080' -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: en-US,en;q=0.9' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36' -H 'Content-Type: application/json' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Referer: http://localhost:8080/nifi/' -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' -H 'DNT: 1' --data-binary '{"revision":{"clientId":"a0f490b8-0165-1000-a521-4d24892f12c0","version":1},"disconnectedNodeAcknowledged":false,"component":{"id":"a0f4dbe2-0165-1000-8cd2-dd50d7dba48d","state":"ENABLED"}}' --compressed
Add a controller service available to a component on the canvas
curl 'http://localhost:8080/nifi-api/process-groups/a0f3dfb7-0165-1000-8310-4fb72628adaa/controller-services' -H 'Origin: http://localhost:8080' -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: en-US,en;q=0.9' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36' -H 'Content-Type: application/json' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Referer: http://localhost:8080/nifi/?processGroupId=root&componentIds=a0f7e36a-0165-1000-ef42-c59f18644d42' -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' -H 'DNT: 1' --data-binary '{"revision":{"clientId":"a0f490b8-0165-1000-a521-4d24892f12c0","version":0},"disconnectedNodeAcknowledged":false,"component":{"type":"org.apache.nifi.ssl.StandardRestrictedSSLContextService","bundle":{"group":"org.apache.nifi","artifact":"nifi-ssl-context-service-nar","version":"1.8.0-SNAPSHOT"},"name":"StandardRestrictedSSLContextService"}}' --compressed
I am trying to retrieve the search results from virustotal for ransomware keyword, where I am trying to use the following code:
wget "https://www.virustotal.com/#/search/ransomware" -O output.txt
However, I do not get the result output. I am expecting to get at least the hash value of each resulted sample, where I am planning to use them later on.
Any advice?
You can use either curl or wget to do this.
Using Curl
curl 'https://www.virustotal.com/ui/search?query=ransomware&relationships\[url\]=network_location%2Clast_serving_ip_address&relationships\[comment\]=author%2Citem' -H 'pragma: no-cache' -H 'cookie: _ga=GA1.2.1407677727.1526818525; _gid=GA1.2.163855722.1526818525; _gat=1' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36' -H 'accept: application/json' -H 'cache-control: no-cache' -H 'authority: www.virustotal.com' -H 'referer: https://www.virustotal.com/' --compressed
Using wget
wget 'https://www.virustotal.com/ui/search?query=ransomware&relationships\[url\]=network_location%2Clast_serving_ip_address&relationships\[comment\]=author%2Citem' -O output.txt
I managed to get the curl output of a web-page that using site minder authentication by manually extracting the cookie from firefox and using it in the curl command:
curl "example.com/firstexample" -H "Host: example.com" -H "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:50.0) Gecko/20100101 Firefox/50.0" -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" -H "Accept-Language: en-US,en;q=0.5" --compressed -H "Content-Type: application/x-www-form-urlencoded" -H "X-Requested-With: XMLHttpRequest" -H "Referer: http://example.com" -H --cookie-jar cookies.txt --cookie cookies.txt -H "Connection: keep-alive"
However, the curl works only as long as the session lasts. Once a new session starts i.e. the cookie session id and other stuff changes - the curl no longer works obviously.
Is there a way to get the subsequent cookies/session ids via curl to continue seemlessly.
Check out the --cookie and --cookie-jar options on curl in the man page. They should do what you need.
I have tried to export a text file resulted by an request to a .jsp page with Java script enabled from Oracle BI Publisher framework with below command in Unix:
curl 'http://server/xmlpserver/report.xdo&_xt=New+Template+1&_xf=text'
-H 'Accept-Encoding: gzip, deflate, sdch' -H 'Accept-Language: en-US,en;q=0.8'
-H 'Upgrade-Insecure-Requests: 1' -H 'User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 Safari/537.36'
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8'
-H 'Referer: http://server/xmlpserver/report.xdo'
-H 'Cookie:
JSESSIONID=qq!11!22; ORA_XDO_UI=en_US;
JSESSIONID=qq!11; oracle.uix=0^^GMT+3:00^p'
-H 'Connection: keep-alive'
--compressed -o test.txt
This command is 'copy as bash curl command' from Google Chrome debugger and work it but syntax command use id JSESSION in cookie who will expire.
I need to login on this .jsp page to generate cookie with JSESSION=qq!11!22 and JSESSION=qq!11 (see above cUrl syntax from Google Chrome).
For this I have used command:
curl -c cookies.txt -d "user=user&pass=pass" http://server/xmlpserver/login.jsp, and was generated only one "JSESSIONID=qq!11;".
I need to generate one more JSESSIONID=qq!11!22.
Thanks!