Instead of writing a PyEnvironment and batching it (using BatchedPyEnvironment), I would like to write a PyEnvironment that is directly in a batched format.
This means my observation should be of shape (batch_size, ..), and my actions, discounts and rewards should be of length batch_size. This would speed up parallelization for my use-case as I can vectorize the evolvement of the state.
Is this possible in tf_agents, and can agents work with such an environment (after converting it into a TFEnvironment)? If so, are there any examples how this can be achieved?
I tried defining the action_spec and observation_spec accordingly, but I don't know how to adjust discount and reward shapes and if batching of individual elements within a time-step does make sense.
self._action_spec = array_spec.BoundedArraySpec(
shape=(self._batch_size,), dtype=np.int32, minimum=0, maximum=1, name='action')
self._observation_spec = array_spec.BoundedArraySpec(
shape=(self._batch_size, 1), dtype=np.int32, minimum=0, name='observation')
Thank you!
Related
I've been trying to figure out if it's at all possible to create a smart contract in Solana that will run a long-ish calculation (e.g. many sha256 iterations, or a KDF), maybe in parts, even though the calculation as a whole is obviously longer than the maximum compute budget for a transaction.
A CPI will not cut it since, as the docs say, the budget is transferred to the other program.
Any help on this issue would be appreciated. Thanks!
Well, that would mean structuring the programs instruction set to be a bit more orchestration.
You can then submit multiple transactions thereby 'chunking' up the CU consumption.
We did something similar to handle both data chunking and distributing the CU to process (validate proofs, etc.).
If you want to use CU for more than 200k(maximum is 1.4m) you can use this trick:
const additionalComputeBudgetInstruction =
ComputeBudgetProgram.requestUnits({
units: 400000,
additionalFee: 0,
});
use the code above before calling your instruction inside ts test, then use it inside your instruction call like this:
await program.methods
.test()
.accounts({
//accounts
})
.signers()
.preInstructions([additionalComputeBudgetInstruction])
.rpc();
hope it helps.
I have a question about cross validation.
In Machine learning, we know there're training, validation, test set.
And test set is final run to see how the final model/classifier performed.
But in the process of cross validation:
we are splitting data into training set and testing set(most tutorial used this term), so I'm confused. Do we need to split the whole data into 3 parts: training, validation, test? Since in cross validation we just keep talking about relationship with 2 set: training and the other.
Could someone help clarify?
Thanks
Yep ,it's a little confusing as some material uses CV/test interchangeably and some material does not use ,but i'll try to make it easy to understand by giving the comprehension of why it's needed:
You need the train set to do exactly that, train, but then also you need a way to ensure that your algorithm isn't memorizing the train set(that it's not overfitting) and how well its doing, so that makes the need of the test set so you can give it data it has never seen and you can measure the performance.
But.... ML its all about experimentation, you will train, evaluate, tweak some knob(hyperparameters or architectures), train again, evaluate again over and over, and then you will select the best experiment results, you deploy your system and in production it gets data it's never seen and it doesn't perform that well ,what happened? You used your test data to fit parameters and make decisions , so you overfitted to this test data but you dont know how it does to data never seen.
Cross validation solves this, you have your train data to learn parameters, and test data to evaluate how it does on unseen data, but still need a way to experiment the best hyper parameters and architectures: you take a sample of your training data and call it cross validation set, and hide your test data , you will NEVER use it until the end.
Now use your train data to learn parameters, and experiment with hyper parameters and architectures, but you will evaluate each experiment on the cross validation data instead of test data(you can see it as using CV data as a way to learn the hyperparameters) , after you experimented a lot, and selected your best performing option(on CV), you now use your test data to evaluate how it performs on data it has never seen before deploying it to production.
This is generally an either-or choice. The process of cross-validation is, by design, another way to validate the model. You don't need a separate validation set -- the interactions of the various train-test partitions replace the need for a validation set.
Think about the name, cross-validation ... :-)
I have been reading through documentation and have been able to get Graphite to receive data I have been sending it. I can definitely see the benefit in tracking things like concurrent users and network load.
But now I have been tasked with implementing it on the client to show things like RAM, and CPU usage. In addition to this, I must also track actions (users buying things). Maybe I am missing a large chunk of the picture here, but I am not sure how I would do these things. Do I need a timestamp? I've also seen plugins for pie charts and such and this would indicate I could perhaps create graphs from devices with different statistics.
What am I missing?
I don't think you're missing anything.
Any data you send on to, say, InfluxDB (as that's what I've used the most) will be timestamped automatically when it arrives - unless you specify an explicit one yourself.
If you're showing, for example, CPU load you can write your query to pick up the latest value, or perhaps an average (mean value) over time, or the max or min over a period of time as appropriate.
Pie charts can also be used successfully to graph relationships over time.
The key is to create a specific query (I use SQL directly) to craft the data used for the panel type. There are excellent examples in the documentation.
I am working on a small project which progressively grows a list of links and then processes them through a queue. There exists the likelihood that a link may be entered into the queue twice and I would like to track my progress so I can skip anything that has already been processed. I'm estimating around 10k unique links at most.
For larger projects I would use a database but that seems overkill for the amount of data I am working with and would prefer some form of in-memory solution that can potentially be serialized if I want to save progress across runs.
What data structure would best fit this need?
Update: I am already using a hash to track which links I have completed processing. Is this the most efficient way of doing it?
def process_link(link)
return if #processed_links[link]
# ... processing logic
#processed_links[link] = Time.now # or other state
end
If you aren't concerned about memory, then just use a Hash to check inclusion; insert and lookup times are O(1) average case. Serialization is straightforward (Ruby's Marshal class should take care of that for you, or you could use a format like JSON). Ruby's Set is an array-like object that is backed with a Hash, so you could just use that if you're so inclined.
However, if memory is a concern, then this is a great problem for a Bloom filter! You can achieve set inclusion testing in constant time and the filter uses substantially less memory than a hash would. The tradeoff is the Bloom filters are probabilistic - you can get false inclusion positives. You can eliminate the probability of most false positives with the right bloom filter parameters, but if duplicates are the exception rather than the rule, you could implement something like:
Check for set inclusion in the Bloom filter [O(1)]
If the bloom filter reports that the entry is found, perform an O(n) check of the input data, to see if this item has been found in the array of input data prior to now.
That would get you very fast and memory-efficient lookups for the common case, and you could make the choice to accept the possibility of false negatives (to keep the whole thing small and fast), or you could perform verification of set inclusion when a duplicate is reported (to only do expensive work when you absolutely have to).
https://github.com/igrigorik/bloomfilter-rb is a Bloom filter implementation I've used in the past; it works nicely. There are also redis-backed Bloom filters, if you need something that can perform set membership tracking and testing across multiple app instances.
How about a Set and convert your links to value object (rather than reference object) like Structs. By creating a value object the Set will be able to detect its uniqueness. Alternately, you could use a hash and store links by their PK.
The data structure could be a hash:
current_status = { links: [link3, link4, link5], processed: [link1, link2, link3] }
To track your progress (in percent):
links_count = current_status[:links].length + current_status[:processed].length
progress = (current_status[:processed].length * 100) / links_count # Will give you percent of progress
To process your links:
push any new link you need to process to current_status[:links].
Use shift to take from current_status[:links] the next link to be processed.
After processing a link, push it to current_status[:processed]
EDIT
As I see it (and understand your question), the logic to process your links would be:
# Add any new link that needs to be processed to the queue unless it have been processed
def add_link_to_queue(link)
current_status[:to_process].push(link) unless current_status[:processed].include?(link)
end
# Process next link on the queue
def process_next_link
link = current_status[:to_process].shift # return first link on the queue
# ... login process the link
current_status[:processed].push(link)
end
# shift method will not only return but also remove the link from the original array to avoid duplications
I've come across a specific task to create a netcdf file which stores data only from certain processors. The matter is the following-I have 3d field, divided into (nx) x (ny) x (nz) domains. Each domain has a processor assigned to it. I would like to save data only from domains in certain position in x direction. This means that the data would come only from ny x nz processors. I've been trying to find examples on how to write such data, but unsucessfully. Does anyone know if this is doable and also-are there specific commands I should use.
For example, I tried to invoke writing data with using if conditions where I used if(mpid%rank==0) then... together with nf90_var_par_access(ncid, varid, nf90_independent) call, but without success, the procedure seems to get stuck.
thank you in advance!
Actually, I've managed to resolve the issue just hours after I posted the question. The main trouble was the dimension length definition in nf90_def_dim call. In it, the code I have assumed by default the dimension, which was a product of number of domains and points in them. I changed that definition to adapt to the case when only certain domains are used and the write process on only few processors worked.
regards to all