My goal is to have appsettings.json file with production configurations and have possibility to costomize it for every developer, e.g. use local connection strings. So it does not similar to transform web.config mechanism, i don't want depends on bulid configuration. Can anyone provide solution for this goal?
In one of my past project we do so: we store all configure information in custom config.xml and parsed it into the custom structure. Web.config contains only server configaration. every developer has own copy of config files with his own data. Solution is that application use configuration files from path, that specified in environment path in windows via Environment.GetEnvironmentVariable("key").
Does anyone have idea better than my one?
This is how I manage configuration: see comments in the code
public Startup(IHostingEnvironment env, IApplicationEnvironment appEnv)
{
var builder = new ConfigurationBuilder()
.AddJsonFile("appsettings.json"); // this one has default configuration
// this file name is added to my gitignore so it won't get committed,
// I keep local dev configuration there
builder.AddJsonFile("appsettings.local.overrides.json", optional: true);
if (env.IsDevelopment())
{
// This reads the configuration keys from the secret store.
// if you need a more secure place for dev configuration use usersecrets
// For more details on using the user secret store see http://go.microsoft.com/fwlink/?LinkID=532709
builder.AddUserSecrets();
}
// the order in which config sources is added is important, a source added later
// will override the same settings from a source added before
// environment variables is usually for production and therefore added last to give it higher priority
builder.AddEnvironmentVariables();
Configuration = builder.Build();
}
Related
I am trying some Spring cloud config server.
I have a scenerio.
if u provide searchlocations = file:///tosomelocation/{profile}
then u have to provide ur applications configuration this way
Inside tosomelocation/dev
app1.yml
app1-dev.yaml
app2.yaml
app2-dev.yaml
application.yaml
But these configurations are picked by applications(app1 and app2) only when they run in dev PROFILE.
Now, If in int PROFILE too, i want app1.yml to be read, then where should i place this app1.yml file ???
right now i am placing app1.yaml in all profile specific folders
int folder
dev folder
e.t.c
Any help on this one ?
Let's say I have connection string for Development environment specified in appsettings.Development.json and connection string for the Staging environment specified in appsettings.Staging.json
All I need to do to switch between Development and Staging is to navigate to Visual Studio Debug tab in project properties and change the value for ASPNETCORE_ENVIRONMENT environment variable.
Now, of course I don't want to have connection string in appsettings.*.json for security reasons. So I move it to User Secrets.
Problem is - it seems there is just one secrets.json file that is used by all the environments. There are no secrets.Development.json or secrets.Staging.json. This means after I switch from Development to Staging environment via Visual Studio Debug tab I then also need to change connection strings manually in secrets.json which kind of defeats the purpose of having built-in support for the environments.
Is this correct that User Secrets are not supported on per-environment basis? If so - is there another approach that would avoid having to modify Secret connection string manually when switching environments?
If you check the tool's parameters with dotnet user-secrets --help you'll see you can specify different secrets per configuration (Debug, Release, any other you want) but not per environment. Which is not a bad decision if you think about it.
The ASPNETCORE_ENVIRONMENT environment variable is meant to tell your application whether the current machine or container is a Development, Production or other environment, so it can pick the appropriate settings file. This environment variable isn't expected to change from one application execution to the next. Even when using containers, the environment variables are passed from the host to the container and aren't expected to change during the container's lifetime.
The secrets files are supposed to be per machine, for development purposes, so there's no need to keep separate files per environment. It makes much more sense to use separate files for configuration, allowing developers to simply change from Dev to Release or Testing or any other custom configuration they may have.
Specifying secrets per configuration
The dotnet user-secrets tool works by reading the UserSecretsId value from the project file and storing the secrets in a JSON file with the same name, eg c952ecfc-344e-43e1-bb67-1ac05973d6c6.json. It's possible to store a UserSecretsId for each configuration.
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|AnyCPU'">
<UserSecretsId>c952ecfc-344e-43e1-bb67-1ac05973d6c6</UserSecretsId>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
<UserSecretsId>7D104000-2230-4EDE-8AE6-63BDDA0BD0C5</UserSecretsId>
</PropertyGroup>
When the -c parameter is used to specify a configuration, the user-secrets tool will read the UserSecretsId value from the corresponding section and use it to store or read secrets.
The dotnet user-secrets init command doesn't recognize the -c parameter, so the csproj file needs to be modified directly.
Once that's done, one can set and read secrets by specifying the configuration, eg :
❯ dotnet user-secrets set -c Debug Key1 Value1
Successfully saved Key1 = Value1 to the secret store.
❯ dotnet user-secrets set -c Release Key1 Value2
Successfully saved Key1 = Value2 to the secret store.
❯ dotnet user-secrets list -c Debug
Key1 = Value1
❯ dotnet user-secrets list -c Release
Key1 = Value2
I also needed this and I think I've come up with an elegant solution.
secrets.json file is shared among all environments you are using, what you can do is add the environment parent to each node in the file and then do the little trick (last 2 code fragments).
Suppose you have a configuration, e.g. in appsettings.json or appsettings.{environment}.json:
{
"Key1": "value1",
"Secret1": "<set yourself>"
}
and then you have the secret part in secrets.json:
{
"Secret1": "my secret value"
}
you can get and bind the whole section easily:
IConfiguration configuration = new ConfigurationBuilder()
.AddJsonFile("appsettings.json")
.AddJsonFile($"appsettings.{environment}.json")
.AddUserSecrets<Program>()
.Build();
var myConfiguration = configuration.Get<MyConfiguration>();
Now comes the problem, I want to have multiple environments in the secrets.json. I personally used to have secrets for all environments I needed and just comment/uncomment what I wanted, however, it's manual work. So I will prefix them with environment name instead.
{
"Development:Secret1": "my secret development value",
"Staging:Secret1": "my secret staging value"
}
You have to load the environment-specific configuration(s) from the IConfiguration instance and override the existing myConfiguration values using:
configuration.GetSection(environment).Bind(myConfiguration);
\\ or
configurationRoot.Bind(environment, configuration);
And that's it.
If you run it using environment="Development", you will have "my secret development value" loaded. If you run it using environment="Staging", you will have "my secret staging value" loaded.
Additional details
The double dot character (:) acts as a new section, so if you write
{
"Development:Secret1": "my secret development value",
"Staging:Secret1": "my secret staging value"
}
it's the same as
{
"Development":
{
"Secret1": "my secret development value"
},
"Staging":
{
"Secret1": "my secret staging value"
}
}
the trick is about loading just the environment-specific part and binding it to the myConfiguration instance.
I must mention that no matter what environment you are using, all the secrets are actually loaded in the memory.
The Secret Manager (https://learn.microsoft.com/en-us/aspnet/core/security/app-secrets?view=aspnetcore-3.1) is designed strictly for development, not any other stage (environment), since it is inherently insecure (local dev secrets are not encrypted). See the warning on the page linked. So there is no need to have per environment secrets storage vis-a-vis that tool. For other environments (staging, prod, etc), Microsoft would likely steer you toward their secure secrets storage service -- Key Vault. You can use the Secret Manager for dev secrets and then store the other environments in Key Vault. I have done this in many Asp.Net Core apps and it works well. For Key Vault info, see this:
https://learn.microsoft.com/en-us/aspnet/core/security/key-vault-configuration?view=aspnetcore-3.1
There is no "out of the box" way to use different secrets per environment. However you can use the options pattern to bind your configs depending on environment:
Create a secrets.json with your environments at the top level:
{
"Local": {
"Secret1": "local secret",
"ConnectionStrings": {
"DB": "LocalDBConnectionstring"
}
},
"Development": {
"Secret1": "dev secret",
"ConnectionStrings": {
"DB": "DevDBConnectionstring"
}
}
}
Create classes to map your secret configurations:
public class SecretConfigurationEnvironment
{
public string secret1 { get; set; }
public ConnectionStringsConfig ConnectionStrings { get; set; }
}
public class ConnectionStringsConfig
{
public string DB { get; set; }
}
Map the secrets to your classes in your program.cs:
var builder = WebApplication.CreateBuilder(args);
builder.Services.Configure<SecretConfigurationEnvironment>(
builder.Configuration.GetSection(builder.Environment.EnvironmentName));
Use your configs with dependencyInjection whereever you need them:
public class MyClass {
private readonly IOptions<SecretConfigurationEnvironment> _secretConfig;
public MyClass(IOptions<SecretConfigurationEnvironment> secretConfig) {
_secretConfig = secretConfig
}
public void MyMethod() {
var envDBConnectionString = _secretConfig.value.Connectionstrings.DB
}
}
Now your configs will change depending on your environment and you can still use all your configs stored in the appsettings.environment.json files (also with options if you like)
That being said, here is the usual disclaimer: Don't use secrets.json for staging or production secrets.
The code i provided is for .net 6, wep-api. Older versions or different project-types are very similar though and can be found at the microsoft documentation.
If I have two settings files
appSettings.json and appSettings.Development.json
When I use publish from Visual Studio, are both supposed to be copied to the target folder? I'm not sure, because they both show up in the target folder (on a dev server) when I publish. I was under the impression that they combined at build time and ONLY the appSettings.json file was published. If not, then do I need to consider manually coding for these differences as Ive seen in a few examples ?
eg. This example is loading the settings via code (NOT how Im doing it)
Note - they are using the environment name, ASPNETCORE_ENVIRONMENT setting
public Startup(IHostingEnvironment env)
{
var builder = new ConfigurationBuilder()
.SetBasePath(env.ContentRootPath)
.AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
.AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true, reloadOnChange: true)
.AddEnvironmentVariables();
Configuration = builder.Build();
}
Some of my Startup class is shown below.
Note: I am not referencing the environment setting.
public Startup(IConfiguration configuration)
{
Configuration = configuration;
}
public IConfiguration Configuration { get; }
// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_2);
// In production, the Angular files will be served from this directory
services.AddSpaStaticFiles(configuration =>
{
configuration.RootPath = "ClientApp/dist";
});
}
// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
else
{
app.UseExceptionHandler("/Error");
// The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
app.UseHsts();
}
app.UseHttpsRedirection();
app.UseStaticFiles();
app.UseSpaStaticFiles();
app.UseMvc(routes =>
{
routes.MapRoute(
name: "default",
template: "{controller}/{action=Index}/{id?}");
});
app.UseSpa(spa =>
{
// To learn more about options for serving an Angular SPA from ASP.NET Core,
// see https://go.microsoft.com/fwlink/?linkid=864501
spa.Options.SourcePath = "ClientApp";
if (env.IsDevelopment())
{
spa.UseAngularCliServer(npmScript: "start");
}
});
}
[ Update ]
I found my answer here - the key I was missing was updating the csproj file for the publish settings related to environment.
https://learn.microsoft.com/en-us/aspnet/core/host-and-deploy/aspnet-core-module?view=aspnetcore-2.2#configuration-with-webconfig
So I assume that if I have several different environments, each with its own settings file, that a publish would result in putting ALL of them out to the target dir?
It's a bit confusing with ASP.NET Core, especially if you're coming from having worked with ASP.NET previously. The build configurations (Debug, Release) really have no bearing on anything that happens with ASP.NET Core. An ASP.NET Core app is technically environment-agnostic. Whereas with an older ASP.NET app, you'd have to publish for a specific environment, you can theoretically take the same ASP.NET Core publish and run it in any of your environments. This is of course aided by the fact that Web.config is not utilized by ASP.NET Core.
This, then, is the reason why all the environment-specific JSON files come along for the ride. Which is ultimately used is based on the value of the ASPNETCORE_ENVIRONMENT environment variable set at runtime, not which build configuration you chose when publishing. Which is actually really nice when you think about it. You can take the same published app, run it in your "staging" environment to ensure everything is working and then deploy it to your "production" environment, simply by ensuring that each environment has the appropriate value for ASPNETCORE_ENVIRONMENT set. This makes release pipelines trivial.
That said, it's still possible to use things like the #if DEBUG compiler directives, and if you do that then there will be differences in your ASP.NET Core app depending on the build configuration chosen, but you should really avoid doing that in the first place. In general, you should rely only on the IHostingEnvironment abstraction in an ASP.NET Core app to determine what happens in what environment.
I am currently setting up a spring cloud config server to deliver some JSON config files from a specific repository, using RabbitMQ to propagate change events to the clients.
Therefore I added a REST endpoint, which delivers all config files, based on a passed branch name:
#RestController
public class RPConfigsEndpoint {
#Autowired
private JGitEnvironmentRepository repository;
private File[] files;
#RequestMapping(value = "/myConfigs")
public File[] getList(#RequestParam(defaultValue = "master") String branch) {
//load/refresh the branch
repository.refresh(branch);
try {
FileRepositoryBuilder builder = new FileRepositoryBuilder();
Repository repo = builder.setGitDir(repository.getBasedir()).readEnvironment().findGitDir().build();
//only return JSON files
files = repo.getDirectory().listFiles((file, s) -> {
return s.toLowerCase().endsWith(".json");
});
} catch (IOException e) {
e.printStackTrace();
}
return files;
}
}
As expected, I get all the files as plain text files... so far so good.
Now my question:
If I modify one of these files and trigger the '/monitor' endpoint on the server, it generates a RefreshEvent as expected:
.c.s.e.MultipleJGitEnvironmentRepository : Fetched for remote master and found 1 updates
o.s.cloud.bus.event.RefreshListener : Received remote refresh request. Keys refreshed []
This event is sent, as the server notices that the commit ID has changed.
The delta of refreshed keys is empty, as it only looks for environment properties (in .yml or .properties files).
Is there a way to reload the whole config file on the client side as it would be done with single properties? Do I therefore need to adapt the change notification?
With my current approach, I would have to reload the whole branch (40 files) instead of reloading only the modified file...
I found a similar question from 2016 on github, without finding the answer.
Thanks in advance for any hint.
Update
repository.findOne(...) and repository.refresh(...) are not thread safe, which must be garanteed in cloud environment, as the service can be contacted by different apps at the same time.
Possible solutions:
'synchronize' the concerned method call(s)
avoid listing up all the files and request single plain text files, as it works out of the box.
I have a Visual Studio 2015 ASP.NET Core 1.0 project that uses the Google Maps API. Currently, I could hard code the API key in two places:
1) the script tag of a view e.g.
<script src="https://maps.googleapis.com/maps/api/js?key=YOUR_API_KEY_HERE"></script>
2) the config.json file of the project e.g.
{
"Keys": { "GoogleMapsAPIKey": "YOUR_API_KEY_HERE" }
}
Either way the key is exposed when using source control so why not use a Windows Environment Variable?
Within Windows System Properties I added a new environment variable called GoogleMapsAPIKey and pasted in the actual key.
Now how do I use this environment variable in either the script tag or in config.json?
The purpose of the question is to get a general answer on how to use environment variables to hide API keys in such situations.
Add the environment variables configuration provider. Example from here:
public Startup(IHostingEnvironment hostingEnvironment)
{
var builder = new ConfigurationBuilder()
.SetBasePath(hostingEnvironment.ContentRootPath)
.AddJsonFile("config.json")
.AddEnvironmentVariables();
}
If you just want to hide the key during development, you can use user secrets.