I am currently setting up a spring cloud config server to deliver some JSON config files from a specific repository, using RabbitMQ to propagate change events to the clients.
Therefore I added a REST endpoint, which delivers all config files, based on a passed branch name:
#RestController
public class RPConfigsEndpoint {
#Autowired
private JGitEnvironmentRepository repository;
private File[] files;
#RequestMapping(value = "/myConfigs")
public File[] getList(#RequestParam(defaultValue = "master") String branch) {
//load/refresh the branch
repository.refresh(branch);
try {
FileRepositoryBuilder builder = new FileRepositoryBuilder();
Repository repo = builder.setGitDir(repository.getBasedir()).readEnvironment().findGitDir().build();
//only return JSON files
files = repo.getDirectory().listFiles((file, s) -> {
return s.toLowerCase().endsWith(".json");
});
} catch (IOException e) {
e.printStackTrace();
}
return files;
}
}
As expected, I get all the files as plain text files... so far so good.
Now my question:
If I modify one of these files and trigger the '/monitor' endpoint on the server, it generates a RefreshEvent as expected:
.c.s.e.MultipleJGitEnvironmentRepository : Fetched for remote master and found 1 updates
o.s.cloud.bus.event.RefreshListener : Received remote refresh request. Keys refreshed []
This event is sent, as the server notices that the commit ID has changed.
The delta of refreshed keys is empty, as it only looks for environment properties (in .yml or .properties files).
Is there a way to reload the whole config file on the client side as it would be done with single properties? Do I therefore need to adapt the change notification?
With my current approach, I would have to reload the whole branch (40 files) instead of reloading only the modified file...
I found a similar question from 2016 on github, without finding the answer.
Thanks in advance for any hint.
Update
repository.findOne(...) and repository.refresh(...) are not thread safe, which must be garanteed in cloud environment, as the service can be contacted by different apps at the same time.
Possible solutions:
'synchronize' the concerned method call(s)
avoid listing up all the files and request single plain text files, as it works out of the box.
Related
Currently, I am using the FileReadingMessageSource to read files from specific directory using Spring Integration.
#Bean
public MessageSource<File> fileReadingMessageSource() {
FileReadingMessageSource source = new FileReadingMessageSource();
source.setDirectory(new File(/temp));
source.setFilter(new SimplePatternFileListFilter("*.csv"));
source.setUseWatchService(true);
source.setWatchEvents(WatchEventType.CREATE);
return source;
}
Problem, I am facing is the configuration above is able to read the files from temp folder and also other folders inside temp folder. For example - If I create a folder inside the temp like temp/csvfolder and place csv file inside csvfolder, still its able to poll the file .
Can we restrict it to specific folder only?
If you don't use the watch service, it will only look at the top level.
If you want to use a watch service you would need to implement a custom DirectoryScanner.
I am developing a mobile application using Xamarin.Forms
I had the following Home page contains login info:
How can we have the application to automatically save the user name, so that they do not have to type it in each time (as in a browser)?
You can use Properties dictionary in Xamarin.Forms Application class. And let the Xamarin.Forms framework handle persisting user name between app restarts and pausing/resuming your app.
Save user name by writing it to Properties dictionary
var properties = Xamarin.Forms.App.Current.Properties;
if(!properties.ContainsKey("username")
{
properties.Add("username", username);
}
else
{
properties["username"] = username;
}
Then, when your login screen is about to appear (for example in OnAppearing method) check Properties for user name:
var properties = Xamarin.Forms.App.Current.Properties;
if(properties.ContainsKey("username")
{
var savedUsername = (string)properties["username"];
}
If it's not there, then it means that this is first time when user log in into your application.
A very similar question was posed just a few days ago - my answer on that question also applies to your question: The best way to save Configuration data in Xamarin.Forms based app?
Essentially, you want to store the information using the native settings functionality. I would advise against using Application.Properties for now. It is currently not reliable on Android, and in the past has had other problems. The nuget package referenced in my linked answer is a better approach and will save you some headache in the future.
The right way to be done is through the App settings plugin
https://github.com/jamesmontemagno/Xamarin.Plugins/tree/master/Settings
What i did in my application is.
1) Installed Plugin.Settings from nuget
2)Added to Helpers->Settings.cs (autogenerated file by plugin) the following
public static class Settings
{
private static ISettings AppSettings
{
get { return CrossSettings.Current; }
}
private const string UserNameKey = "username_key";
private static readonly string UserNameDefault = "demo";
public static string UserName
{
get { return AppSettings.GetValueOrDefault<string>(UserNameKey, UserNameDefault); }
set { AppSettings.AddOrUpdateValue<string>(UserNameKey, value); }
}
}
3)In order to keep the username in the Application Context set
Settings.UserName = ViewModel.Username;
4)When you login screen starts
string username = Settings.UserName;
The answer is simple: persistance. Servers do this by setting cookies containing the data (or reference to it) that they want you to see when rendering the form field.
In order to do this in an app (with Xamarin for instance), you need to store the user's data into a file or database somewhere. Since you're using Xamarin you can probably use some sort of ConfigurationManager to keep track of this.
Obviously you could just create a config file in the local storage you have for your app (I don't think you need permissions to create files in that space).
When you have the info stored somewhere, just retrieve it and set the input's value to it.
My goal is to have appsettings.json file with production configurations and have possibility to costomize it for every developer, e.g. use local connection strings. So it does not similar to transform web.config mechanism, i don't want depends on bulid configuration. Can anyone provide solution for this goal?
In one of my past project we do so: we store all configure information in custom config.xml and parsed it into the custom structure. Web.config contains only server configaration. every developer has own copy of config files with his own data. Solution is that application use configuration files from path, that specified in environment path in windows via Environment.GetEnvironmentVariable("key").
Does anyone have idea better than my one?
This is how I manage configuration: see comments in the code
public Startup(IHostingEnvironment env, IApplicationEnvironment appEnv)
{
var builder = new ConfigurationBuilder()
.AddJsonFile("appsettings.json"); // this one has default configuration
// this file name is added to my gitignore so it won't get committed,
// I keep local dev configuration there
builder.AddJsonFile("appsettings.local.overrides.json", optional: true);
if (env.IsDevelopment())
{
// This reads the configuration keys from the secret store.
// if you need a more secure place for dev configuration use usersecrets
// For more details on using the user secret store see http://go.microsoft.com/fwlink/?LinkID=532709
builder.AddUserSecrets();
}
// the order in which config sources is added is important, a source added later
// will override the same settings from a source added before
// environment variables is usually for production and therefore added last to give it higher priority
builder.AddEnvironmentVariables();
Configuration = builder.Build();
}
I have my web application using Play 2.3.3. Here is a method in my controller :
#Cached(key = "categories", duration = 3600)
public static Result index() throws ApiException {
// Stuff read from database here
return ok(MyView.render(data));
}
My question is how can I delete the categories element from the cache manually to force it to be regenerated ? For example, when I add a category in my database, I don't want to wait for the cache to expire to make it visible to my users.
I couldn't find anything in the documentation : https://playframework.com/documentation/2.3.x/JavaCache
So is the cache stored in RAM, somewhere in the filesystem ?
Thanks.
EDIT : Ideally, I'd like to connect to my server, browse to a directory and delete the file. I suppose that's not possible with EHCache ?
You would use Cache.remove(key) from the docs you have linked.
import play.api.cache.Cache
Cache.remove("categories")
Play uses EhCache by default, which is an in-memory cache really only suited for single machine deployments. You can replace it another cache plugin though, play2 memcached for example.
I have Visual Studio 2010 solution with an Azure Service and an ASP.NET MVC 3 solution that serves as a Web Role for the Azure service. No other roles attached to the service other than that.
Every deployment to the Azure staging (or production, for that matter) environment takes up to 20 minutes to complete, form the moment I click publish on Visual Studio until all instances (2) are started.
As you can imagine this makes it a PITA to publish often, or to quick-fix some bugs. Is there a way to speed the process up? Would it be faster to upload the package to de Blob storage and upgrade from there? How would I go about achieving that?
I feel on-line docs on Azure leave a lot to be desired. Particularly when it comes to troubleshooting by the way.
Thanks.
One idea for reducing the need (and frequency) for redeploying is to move static content into blob storage, external to the package. For instance, move your css and javascript to blob storage, along with images. Once this is done, you'd only have to recompile / redeploy for .NET code changes. You can upload updated css, at any time, to blob storage. If you want to test this in staging first, you could always have a staging vs. production container name for your static content and store that container name in a config setting.
This doesn't change the deployment time when you do need to redeploy, but at least you can reduce how often you go through that process...
You should enable Web Deploy in your Azure project. It works this way :
1/ Create a RDP account (don't forget, you need to upload a certificate with its private key so that Azure can decipher the password). That is hidden in the Deploy Dialog Box for your Azure deployment project.
2/ Enable Web Deployment - same place
Once you've published the app that way, right-click in the web application (not the azure deployment project) and select Publish. The pop-up has everything defined except the password, enter that as well and you'll upload your changes to Azure in a matter of seconds.
CAVEAT : this is meant for single-instance web apps, definitely not the way to go for a production upgrade strategy, and the Blob storage answer already mentioned is the best option in that case.
Pierre
My solution to this problem is only to push a new package when I am changing code in the RoleEntryPoint or with the Service Definition. In Azure 1.3 you now have the ability to use Remote Desktop Connection. Using RDC, I will compile my code locally and use copy/paste to place it on the Azure server in the appropriate directory. Once the production code is running correctly, I can then push the fully tested version to staging and then do a VIP swap. This limits the number of times I actually have to deploy a package.
You actually have quite a long window in which you can keep modifying your code in Azure before you have to publish a new package. The new package is only really needed for those cases where Azure has to shutdown/restart your role instance.
It's a nice idea to try uploading your project to blob storage first, but unfortunately this is what Visual Studio is doing for you behind the scene anyway. As has been pointed out elsewhere, most of the time in doing the deploy is not the upload itself, but the stopping and starting of all of your update domains.
If you're just running this site in a development environment, then the only way I know to speed it up is to run just one instance. If this is the live environment, then... sorry, I think you're out of luck.
So that I don't have to deploy to the cloud to test minor changes, what I've found works quite well is to engineer the site so that it works when running in local IIS just like any other MVC site.
The biggest barrier to this working are settings that you have in the cloud config. The way we get around this is to make a copy of all of the settings in your cloud config and put them in your web.config in the appSettings. Then rather than using RoleEnvironment.GetConfigurationSettingValue() create a wrapper class that you call instead. This wrapper class checks RoleEnvironment.IsAvailable to see if it is running in the Azure fabric, if it is, it calls the usual config function above, if not, it calls WebConfigurationManager.AppSettings[].
There are a few other things that you'll want to do around getting the config setting change events which hopefully you can figure out from the code below:
public class SmartConfigurationManager
{
private static bool _addConfigChangeEvents;
private static string _configName;
private static Func<string, bool> _configSetter;
public static bool AddConfigChangeEvents
{
get { return _addConfigChangeEvents; }
set
{
_addConfigChangeEvents = value;
if (value)
{
RoleEnvironment.Changing += RoleEnvironmentChanging;
}
else
{
RoleEnvironment.Changing -= RoleEnvironmentChanging;
}
}
}
public static string Setting(string configName)
{
if (RoleEnvironment.IsAvailable)
{
return RoleEnvironment.GetConfigurationSettingValue(configName);
}
return WebConfigurationManager.AppSettings[configName];
}
public static Action<string, Func<string, bool>> GetConfigurationSettingPublisher()
{
if (RoleEnvironment.IsAvailable)
{
return AzureSettingsGet;
}
return WebAppSettingsGet;
}
public static void WebAppSettingsGet(string configName, Func<string, bool> configSetter)
{
configSetter(WebConfigurationManager.AppSettings[configName]);
}
public static void AzureSettingsGet(string configName, Func<string, bool> configSetter)
{
// We have to store these to be used in the RoleEnvironment Changed handler
_configName = configName;
_configSetter = configSetter;
// Provide the configSetter with the initial value
configSetter(RoleEnvironment.GetConfigurationSettingValue(configName));
if (AddConfigChangeEvents)
{
RoleEnvironment.Changed += RoleEnvironmentChanged;
}
}
private static void RoleEnvironmentChanged(object anotherSender, RoleEnvironmentChangedEventArgs arg)
{
if ((arg.Changes.OfType<RoleEnvironmentConfigurationSettingChange>().Any(change => change.ConfigurationSettingName == _configName)))
{
if ((_configSetter(RoleEnvironment.GetConfigurationSettingValue(_configName))))
{
RoleEnvironment.RequestRecycle();
}
}
}
private static void RoleEnvironmentChanging(object sender, RoleEnvironmentChangingEventArgs e)
{
// If a configuration setting is changing
if ((e.Changes.Any(change => change is RoleEnvironmentConfigurationSettingChange)))
{
// Set e.Cancel to true to restart this role instance
e.Cancel = true;
}
}
}
The uploading itself takes a bit more than a minute most of the time. It's the starting up of the instances that take up most of the time.
What you can do is to deploy your fixes to staging first (note that it costs money so don't let it be there for too long). Swapping from staging to production only takes a couple of seconds. So while your application's still running you can upload the patched version, let your testers test it on staging and when they give the go then simply swap it to production.
I haven't tested your possible alternative approach by first uploading to blob storage first. But I think that's overhead as it doesn't speed up starting up the instances.