Validating configurations files with viper - go

I was looking for a configuration parser for go and https://github.com/spf13/viper seems to come highly recommended.
I am very surprised to find that configuration files are not validated by default.
Viper parses files and extracts requested values from them but I cannot find a way to detect bad configuration.
For instance I if create a (Java style) .properties file containing just "???" and nothing else. This is accepted without any error.
I can understand the philosophy that you should ignore irrelevant configuration items but I desire more rigor. I would also like to reject anything that does not match the X=Y format in a properties file.
To me this is a fatal flaw that suggests I should use a different package (or roll my own as usual).
Have I missed something? Does viper in fact support detecting and rejecting bad configuration keys?

I think the answer is no. viper does not validate java .properties files.
I posted a bug report (or feature request depending on your point of view) as https://github.com/spf13/viper/issues/790

You can try https://github.com/num30/config library which is based on Viper. It has built-in validation.

Related

I am looking for code policy enforcement tool for xml and Python

I have projects that are developed with xml and python code mostly (Odoo modules). There is a bit of .po files for translation and csv fields for data.
I would like to enforce specific policies in xml files, for example:
No duplicate id attributes.
A specific attribute must be present if child elements contain a specific tags.
On python, I want to enforce rules like:
Look for SQL queries, and make sure that they use specific parameter methods to prevent SQL injection
Follow a specific naming convention
Some attributes are required in classes that inherit a specific class
I hope that the idea is clear.
Is there any open source solution for this? Preferably linked with github and checks on every commit!
I found a python package made specifically for this, pylint-odoo, here.
It can also be installed with pip install pylint-odoo.
An example .pylintrc config file can be found at the web OCA module, here. They also have another file named .pylintrc-mandatory.
There is even a warning for duplicate xml id attribute W7902.

How to plugin into grpc-java to modify code generation and add setNameOrClear(null) methods?

We are having too many issues around all this extra code for every database field with regard to
if(databaseObj.getName() != null)
builder.setName(databaseObj.getName());
and I read square wired into protobuf adding setOrClear methods in java. How do we do this when we generate as well using gradle?
We are using the gradle code from this page right now..
https://github.com/grpc/grpc-java
thanks,
Dean
You can accomplish that via protoc_insertion_points. When you generate the Java code you will see comments like // ##protoc_insertion_point(...). That is where the insertion will occur.
While appearing useful, this approach has serious drawbacks for .protos used in multiple projects. All projects using the same .proto and in the same language should use the same plugins, otherwise it causes the diamond dependency problem. This is why gRPC did not use this approach and instead generates its classes in separate files from the normal message generation. I strongly discourage against this approach, as it paints you into a corner and you don't know when you will need to "pay the piper."
To insert into a point, your plugin needs to run in the same protoc command-line invocation as the java builtin. Your plugin would then need to set CodeGeneratorResponse.file.insertion_point and content for each file you want to inject code.

In STS 4.0 ignoring unknown property support missing

Following error shown in the problem section for unknown property
There is no easy way to fix the warning in STS4
Even after adding the quickfix, the error is not seems to be going
Description Resource Path Location Type
'IntuitAccountingAPIHost' is an unknown property. application.properties /hk-qbo-connector/src/main/resources line 8 org.eclipse.lsp4e.diagnostic
'OAuth2AppClientId' is an unknown property. application.properties /hk-qbo-connector/src/main/resources line 2 org.eclipse.lsp4e.diagnostic
'OAuth2AppClientSecret' is an unknown property. application.properties /hk-qbo-connector/src/main/resources line 3 org.eclipse.lsp4e.diagnostic
'OAuth2AppRedirectUri' is an unknown property. application.properties /hk-qbo-connector/src/main/resources line 5 org.eclipse.lsp4e.diagnostic
You should probably consider raising your problem as a bug-report or feature request on our issue tracker. It doesn't read like a 'question' to me, and so it probably doesn't belong on SO as such.
Nevertheless let me try and answer as best as I can.
Even after adding the quickfix, the error is not seems to be going
It will, once you save the additional-spring-configuration-metadata.json file where the quickfix creates new metadata for you.
I agree it is a little unintuitve that the change is not saved automatically, but I'm afraid we (STS vscode extension) don't have control over this. The changes are being applied by the Vscode language server client, and it's the client's decision to apply the changes only to 'dirty' editor buffers rather than apply them directly to the files on disk.
As to your general statement that 'In STS 4.0 ignoring unknown property support is missing'... I think you are referring to the fact that in STS3 it used to be possible to ignore problems of a given type altogether and this is no longer possible in STS4. This is true. If you like to get this kind of user-configurability back again, please consider raising a feature request on our issue tracker. SO, really isn't the place to ask for this sort of thing.
In STS 4.11, you will find this option in Window -> Preferences -> Language Servers -> Spring Language Servers -> Spring Boot Language Servers -> Yaml Editor or Properties Editor.

See parameters that are overridden from TeamCity template

Is there a way to see TeamCity configurations that override parameter defined in template?
I don't think so. What's worked for me in the past was to search through the project files on the filesystem. If you have many build configs, this will be faster than opening each of them in the GUI.
Search for something like this:
<param name="myParamInheritedFromTemplate" value="myOverrideValue" />
in <TeamCity data directory>/config/projects/**/*.xml. If it's absent in an XML file, that build config just inherits the value. If it's present, it overrides it.
It's hacky but it's quick.
There is a feature request https://youtrack.jetbrains.com/issue/TW-21212, please vote. Current workaround are to either search the raw XML files with the settings stored under TeamCity Data Directory on the server as #sferencik suggested, or use REST API to get settings of all the build configurations and search for the parameter there. Let me know if you need help on any of these.

Output all language strings in Revel?

I'm developing an API Server in Go and the server (at the moment) handles all translations for clients. When an API client fetches particular data it also asks for the translations that are available for the given section.
Ideally I want to have the following folder structure:
/messages
/home.en
/home.fr
/home.sv
/news.en
/news.fr
/news.sv
Where news and home are distinct modules.
Now the question I have for Revel is is it possible to fetch ALL language strings for a given module and given locale? For example pull all home strings for en-US.
EDIT:
I would like the output (something I can return to the client) a key:value string of translations.
Any guidance would be appreciated.
It seems to me that revel uses messaged based translation (just like gettext does), so you need
the original string to get the translation. These strings are stored in Config objects,
which are themselves stored in messages of i18n.go, sorted by language.
As you can see, this mapping is not exported, so you can't access it. The best way
to fix this is to write a function for what you want (getting the config by supplying a language)
or exporting one of the existing functions and create a pull request for revel.
You may workaround this by copying the code of loadMessageFile or by forking your version
of revel and exporting loadMessageFile or parseMessagesFile. This also is a great opportunity
to create a pull request.
Note that the localizations are stored in a INI file format parsed by robfig/config,
so manually parsing is also an option (although not recommended).

Resources