I thought Solana/Metaplex etc should be able to handle large numbers of transactions in quick succession. I just wrote a load test to do 50 mints of an existing SPL token (that has metaplex token-data associated with it)
In my code I dont specify any particular node/rpc - rather just the cluster i.e. testnet
What should I be doing here ?
{"name":"Error","message":"failed to get info about account 2fvtsp6U6iDVhJvox5kRpUS6jFAStk847zATX3cpsVD8: Error: 429 Too Many Requests: {\"jsonrpc\":\"2.0\",\"error\":{\"code\": 429, \"message\":\"Too many requests from your IP, contact your app developer or support#rpcpool.com.\"}, \"id\": \"76627a31-4522-4ebb-ae22-5861fa6781f0\" } \r\n","stack":"Error: failed to get info about account 2fvtsp6U6iDVhJvox5kRpUS6jFAStk847zATX3cpsVD8: Error: 429 Too Many Requests: {\"jsonrpc\":\"2.0\",\"error\":{\"code\": 429, \"message\":\"Too many requests from your IP, contact your app developer or support#rpcpool.com.\"}, \"id\": \"76627a31-4522-4ebb-ae22-5861fa6781f0\" } \r\n\n at Connection.getAccountInfo (/Users/ffff/dev/walsingh/TOKENPASS/tpass-graphql/graphql/node_modules/#metaplex/js/node_modules/#solana/web3.js/lib/index.cjs.js:5508:13)\n at runMicrotasks (<anonymous>)\n at processTicksAndRejections (node:internal/process/task_queues:96:5)\n at async Token.getAccountInfo (
The 429 issue you are running into is a RPC rate limit. Testnet has the following rate limits at the time of writing:
Maximum number of requests per 10 seconds per IP: 100
Maximum number of requests per 10 seconds per IP for a single RPC: 40
Maximum concurrent connections per IP: 40
Maximum connection rate per 10 seconds per IP: 40
Maximum amount of data per 30 second: 100 MB
You probably ran into one of these limits. The general recommendation is to go get access to one of the RPCs without rate limits, as the public endpoints are not meant for testing how many transactions you can get through.
Quicknode, Triton, and Genesysgo provide RPC infra to use.
Related
I'm using Google Sheet API in 4 different apps.
In 2 of them I'm getting this error far before I actually reach the limit:
GSheet error: rateLimitExceeded: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per user per 100 seconds' of service 'sheets.googleapis.com' for consumer 'project_number:xxx'.
I also get the same error but for writing.
You can see here, my limits are 500 requests per 100 seconds, and I max out around 15, yet I still get this error a LOT.
Any ideas?
I have jmeter , where a single thread contains two mqtt gateway connection sampler & each sampler have three publishers connected to iothub.
Jmeter reference:
When I run the thread in loop 6frames / second for 10 seconds, I could see all 60 frames published successful in JMeter.
But when I check data count at iothub, first gate way point have received only 6 frames ( some data get missed it seems, problem with jmeter I assume ) & second gateway have received 42 frames. Second part led to major confusion, when it have to receive maximum of 30 frames, but received 42.
Diagram reference:
Each gateway (A &B) include the Connection panel with :
Iothub URL
Mqtt v 3.1.1
Username: iothuburl/device ID
Pwd: SAS token ( generated SAS from connection string available at iothubowner page from azure portal).
Each Gateway (A&B) include
three publishers & Each includes 200 JSON objects and size doesn't exceed 55kb.
Publisher QoS: 0
Operation:
For every one second, each gateway publish 3 frames ( total 600 JSON objects).
As I have mentioned 2 gateway, so total 6 frames with 1200 JSON object get published successful in JMeter.
But the data is missing at iothub.
note: while running two gateway in single thread, i could both gateway connection sharing the common connection string ID.
Any clue, where did I miss the major configuration, any help would be greatly appreciated. Thanks.
Change the QoS=1 in publisher panel. Though we have few latency time to wait for acknowledgement, but the simulation works fine without any loss of connection/data.
I have created API using Django Rest Framework.
API communicates with GCP cloud storage to store profile Image(around 1MB/pic).
While performing load testing (around 1000 request/s) to that server.
I have encountered the following error.
I seem to be a GCP cloud storage max request issue, but unable to figure out the solution of it.
Exception Type: SSLError at /api/v1/users
Exception Value: HTTPSConnectionPool(host='www.googleapis.com', port=443): Max retries exceeded with url: /storage/v1/b/<gcp-bucket-name>?projection=noAcl (Caused by SSLError(SSLError("bad handshake: SysCallError(-1, 'Unexpected EOF')",),))
Looks like you have the answer to your question here:
"...buckets have an initial IO capacity of around 1000 write requests
per second...As the request rate for a given bucket grows, Cloud
Storage automatically increases the IO capacity for that bucket"
Therefore it automatically Auto-Scale. The only thing is that you need to increase the requests/s gradually as described here:
"If your request rate is expected to go over these thresholds, you should start with a request rate below or near the thresholds and then double the request rate no faster than every 20 minutes"
Looks like your bucket should get an increase of I/O capacity that will work in the future.
You are actually right in the edge (1000 req/s), but I guess this is what is causing your error.
I am getting the Socket Exception while running load test on self-provisioned test rig.
I am trigger those load tests in agent machine(self-provisioned test rig) from my local machine.
Note : For first 2 to 3 minutes test iterations are passing , after that we are getting the Socket Exception.
Below is the error message :
A connection attempt failed because the connected party did not
properly respond after a period of time, or established connection
failed because connected host has failed to respond.
Below are the stack trace details :
at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult) at
System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure,
Socket s4, Socket s6, Socket& socket, IPAddress& address,
ConnectSocketState state, IAsyncResult asyncResult, Exception&
exception)
Run Time - 20min
Sample rate - 10sec
warm up duration 10sec
number of agents used - 2
Load pattern :
initial load - 10user
max user count - 300
step duration - 10sec
step user count - 10
Although, by Changing above values I am still getting the exception in the same way.
I am using Visual studio 2015 enterprise.
The question states: start with 10 users, every 10 seconds add 10 users to a maximum of 300. Thus after 29 increments there will be 300 users and that will take 29*10 seconds which is 4m50s. The test will thus (attempt to) run with the maximum load of 300 users for the remaining 15m10s.
Given that all tests pass for the first 2 or 3 minutes plus the the error message, that suggests that you are overloading some part of the network. It might be the agents, it might be the servers or it might be on the connections between them. Some network components have a maximum number of active connections and the 300 users might be too many.
Increasing the load so rapidly means you do not clearly know what the limiting value. The sampling rate (at 10 seconds) seems high. At each sampling interval a lot of data is transferred (i.e. the sample data) and that can swamp parts of the network. You should look at the network counters for the agents and controller, also the servers if available.
I recommend changing the load test steps to add 10 users every 30 seconds, so it takes about 15 minutes to reach 300 users. It may also be worth reducing the sample rate to every 20 seconds.
We are facing issue sometimes while pulling the information from yammer.
following is the error
https://www.yammer.com/oauth2/access_token.json?client_id=q1gKtZ6UyinT6WWYqTEoag&client_secret=xYKbPYYJLX5U6cyjPuUEdq9DMfKCixXBo7QaGw4QHuU&code=CZnZejAnylQITtsVN7wxVw
[[31merror[0m] play - Cannot invoke the action, eventually got an error: java.lang.RuntimeException: Failed : HTTP error code : 429 : null
[[31merror[0m] application -
You are exceeding the rate limits when you are pulling data. See below
API calls are subject to rate limiting. Exceeding any rate limits will result in all endpoints returning a status code of 429 (Too Many Requests). Rate limits are per user per app. There are four rate limits:
Autocomplete: 10 requests in 10 seconds.
Messages: 10 requests in 30 seconds.
Notifications: 10 requests in 30 seconds.
All Other Resources: 10 requests in 10 seconds.
https://developer.yammer.com/v1.0/docs/rest-api-rate-limits