We are looking to migrate from delayed_job to resque for our site, and one of the caveats I am seeing all over is that while delayed_job used yaml for serialization, resque uses json, and because of this the hash keys and etc which are symbols need to be accessed as strings, since symbols cannot be stored in json(e.g source).
My question is, if this is true, was there a reason for picking json over yaml? Or rather is there any advantage that json offers for this situation?
There are some JSON support by using redis hashes but I never used it and I doubt it's any useful in this case.
Resque probably uses JSON instead of YAML because it's more broad. I also found a comment on this issue about that.
#defunkt
YAML does not preserve Ruby objects better - with complicated objects
it can serialize an object that it can't unserialize later. This is
unacceptable.
If you want symbol keys I believe YAJL has an option we could use to
preserve the keys. Also you could trivially call symbolize_keys on an
object if you have ActiveSupport.
https://github.com/resque/resque/issues/26
Related
Is there a way to use dictionary rather than using a yaml config for parameters.yml? I want to keep it as a Python Object because my IDE can then track the dependency easily. For my parameters, I am injecting functions in it.
If i need to use yml, I will have to use
def steps1(x, func1):
func1 = eval(func1)
And this will break the refactoring features easily.
You could overwrite the params property in your src.package_name.run.ProjectContext so that it uses a Python dictionary instead of the config loader. You’re also welcome to write up your custom ConfigLoader and use that instead (by overriding _create_config_loader), but that’s probably more effort.
Please bear in mind that parameters in Kedro are meant to be “as dumb as possible” though, as it’s considered static configuration and it’s better separated out of code. What you describe, with expressions, sounds more suited for nodes.
I am looking for a way to emit YAML files avoiding the use of aliases (mostly for simplified human readability). I think extending Psych::Visitors::Emitter or
Psych::Visitors::Visitor is the way to go, but I cannot actually find where Ruby decides whether to dump an anchor in full, or reference it with an alias.
I wouldn't even mind if the anchors were used repeatedly (with their &...... references), I just need to expand aliases to the full structures.
I am aware of similar questions being asked in the past, but:
Ruby YAML write without aliases remained unanswered
Is it possible to emit valid YAML with anchors / references disabled using Ruby or Python? gave answer for Python but not for Ruby
One simple (hacky) approach I used was convert the yaml to json. and then convert it back to YAML. new YAML does not contain aliases/anchors.
require 'json'
jsonObj = oldYaml.to_json
newYaml = YAML.load(jsonObj)
print newYaml.to_yaml
The only way I've found to do this is to perform a deep clone of the object being dumped to YAML. This is because YAML will identify the anchors and aliases based on their identity, and if you clone or dup them, the new object will be equal, but have a different identity.
There are many ways to perform a deep clone, including library support, or writing your own helper function -- I'll leave that as an exercise for the reader.
david4dev's answer to this question claims that there are three equivalent ways to convert an object to a JSON string using the json library:
JSON.dump(object)
JSON.generate(object)
object.to_json
and two equivalent ways to convert a JSON string to an object:
JSON.load(string)
JSON.parse(string)
But looking at the source code, each of them seems to be pretty much different, and there are some differences between them (e.g., 1).
What are the differences among them? When to use which?
TL;DR:
In general:
Use to_json (or the equivalent JSON::generate).
Use JSON::parse.
For some special use cases, you may want dump or load, but it's unsafe to use load on data you didn't create yourself.
Extended Explanation:
JSON::dump vs JSON::generate
As part of its argument signature, JSON::generate allows you to set options such as indent levels and whitespace particulars. JSON::dump, on the other hand, calls ::generate within itself, with specific pre-set options, so you lose the ability to set those yourself.
According to the docs, JSON::dump is meant to be part of the Marshal::dump implementation scheme. The main reason you'd want to explicitly use ::dump yourself would be that you are about to stream your JSON data (over a socket for instance), since ::dump allows you to pass an IO-like object as the second argument. Unfortunately, the JSON data being produced is not really streamed as it is produced; it is created en masse and only sent once the JSON is fully created. This makes having an IO argument useful only in trivial cases.
The final difference between the two is that ::dump can also take a limit argument that causes it to raise an ArgumentError when a certain nesting depth is exceeded.
Comparison to #to_json
#to_json accepts options as arguments, so internal implementation aside, JSON::generate(foo, opts) and foo.to_json(opts) are equivalent.
JSON::load vs JSON::parse
Similar to ::dump calling ::generate internally, ::load calls ::parse internally. ::load, like ::dump, may also take an IO object, but again, the source is read all at once, so streaming is limited to trivial cases. However, unlike the ::dump/::generate duality, both ::load and ::parse accept options as part of their argument signatures.
::load can also be passed a proc, which will be called on every Ruby object parsed from the data; it also comes with a warning that ::load should only be used with trusted data. ::parse has no such restriction, and therefore JSON::parse is the correct choice for parsing untrusted data sources like user inputs and files or streams with unknown contents.
Usually to remove some fields from collections in MongoDB, $unset is used. What can I do in Ruby to accomplish that? I have tried to read this, but I couldn't understand how to use it. I am new to Ruby and still learning to work with it.
The $unset operator is used through the update command which is documented here for the ruby driver.
Taking a step back, the standard (10gen-maintained) MongoDB Ruby driver can do absolutely everything that mongodb can. It exposes very literally the full capabilities of mongodb, just like the mongodb drivers in other languages do. So don't worry about being able to do mongo things in any particular language.
You might be wondering about the capabilities of popular ODM layers since they vary. ODM layers map mongodb documents to native language objects. In Ruby, mongoid is the most popular ODM. As #Tilo points out, it also supports explicitly removing fields from documents through its higher-level API.
Please check this issue for Mongoid:
https://github.com/mongoid/mongoid/pull/635
p = Product.first
p.raw_attributes.delete :foo
p.save
I want to build a SOAP client using ruby. I tried using the soap4r library to generate ruby classes out of the WSDL file, but the issue with this was that all the methods it generated were of optional kind, instead of NAME/VALUE pairs. Given that some methods have a very large number of arguments, many of which are optional, I would prefer to use something like SOAP::Lite(Perl Library) which does not depend on WSDL file and accepts arguments as NAME/VALUE pairs.
Also take a look at Savon.
I've not actually used this myself, but I remembered hearing about it the other day: Handsoap. Check it out and see if it fits your needs! ;)