Can a java TestContainer be started without attempting to call the default registry? - testcontainers

At work, we have decided we would like to use the java TestContainers package, and it has fallen to me to learn and implement this package. It is important to note that security is a top priority at work, so we are behind a proxy and run our own registry to contain our docker images.
This is what I've got so far.
var dockerImage = DockerImageName.parse("<secure-registry>/mongo:latest")
.asCompatibleSubstituteFor("mongo");
MongoDBContainer mongoDBContainer = new MongoDBContainer(dockerImage)
.withImagePullPolicy(PullPolicy.defaultPolicy())
.withExposedPorts(27019);
mongoDBContainer.start();
Not alot, but that's because I can't progress any further. Any time I attempt to run this code, it gets as far as .start() before throwing the following error
com.github.dockerjava.api.exception.InternalServerErrorException: Status 500: {"message":"Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 192.168.65.1:53: no such host"}
The issue is obviously that our proxy is blocking the default registry. I could probably fiddle with proxy settings on my machine so the registry can be resolved, but that is not a company wide solution. This app will be developed and run on many different machines and environments, and reconfiguring all of them would be more effort than it's worth. Furthermore, it shouldn't be communicating with the default registry at all. We have our own registry for a reason. The only real solution I see here is to remove this call entirely but so far, the documentation for both DockerImageName and the containers have been of no help.
If anyone has attempted this, or has another solution, it would be very helpful.

I have since realised my mistake. This page in the documentation reveals the problem. When running, the TestContainers library creates additional containers mainly for management of said containers. Unless a location is specified, it will attempt to pull these from the default registry. The preferred registry can be specified using the instructions in the link above.
As yuppie-flu also mentioned in their comment, this link may provide a nice, global way to set registries.

Related

Unit tests for a OAuth/Config client Spring application

I have a very simple Spring application with cloud config client and oauth resources. It's turning out to be quite a challenge to run the unit tests without requiring the config server to be up.
None of the solutions out there for disabling cloud config client work for the latest version of spring. I have even tried to use empty or bogus config.import urls, but neither disable it. I really dont want to spin up the config server just for running unit tests and it doesnt feel like the correct way either.
What is the recommended way to setup tests for a config client application?
Looking for pointers, ideas.
Thankyou!
Making the configuration optional in test seems to do the job, though you still see an attempt being made to download the config.
spring.config.import: "optional:configserver:http://localhost:8888"
OR you can use
spring.cloud.config.enabled: false
However, client will always attempt to download the configuration, even though it appears to treat it as optional and moves on. I think this is really confusing for new users - one thinks the client is still enabled. What happens when the configserver is up and you expect the client to not download the config, but it still does. I guess the only way to find out is to test.
The weird part is that the following combination fails to load the context.
spring.config.import: "optional:configserver:http://localhost:8888"
spring.cloud.config.enabled: false
Either the import is set to optional or the config is disabled, but not both at the same time.
And, this is only for the tests. Creating another profile application-standalone.yml with same settings as test, fails.

How to shut down the host from within the container?

I'm writing a UPS monitoring console application that will automatically shut down the server when the UPS battery reaches a predefined level of charge during a power outage.
I'd like to run this within a container if possible, as NUT (Network UPS Tools) is notoriously difficult to get working on a Windows system. I've configured it once, on bare metal, and I wish not to have to endure that pain a second time, or worse, a third. (But that's another discussion for another day.)
Back to today: how may I send a system shutdown command to the container's host? I've tried simply shutdown /s /t 0, but that shuts down the container. I need to reach the host.
--NOTE TO CLOSE VOTERS--
This question is not off-topic. Note this Q&A, for one example, which currently bears upvotes numbering well over 2,000. There isn't a single programming concept mentioned in the question.
Docker is a tool used by developers. So its subject matter is very relevant on this forum. Run a quick query on the docker tag and you'll see what I mean.
An approach that involves a little bit of programming would be to create a small service that sits on the host and waits for incoming connections.
That service could then either execute the shutdown command or call the appropriate Windows API call.
I suggest giving this service a minimal http endpoint with a single URL and protect it with user / password and SSL/TLS.
You would then call this from the container using curl --user user:pass https://host/shutdown
It should be fairly easy to implement this with golang and nssm.
psshutdown \\host -u ... -p ... should do the trick, assuming you are able / willing to store credentials with sufficient permissions in the container.
See https://technet.microsoft.com/en-us/sysinternals/psshutdown.aspx for more details and a download link.
Edit: I leave this here for documenting an approach that doesn't work because of changes in the way Windows handles security. Follow InteXXs link provided below, it clearly describes what would be necessary to get psshutdown working again and which security risks this brings.

Windows API Clear Authentication Tokens

I'm using the WNetEnumResource to enumerate all network share connections and WNetCancelConnection2 to close them. Then I am using WNetUseConnection to connect to a share using discrete credentials. This process happens multiple times throughout the day.
The problem that I'm running into is that after the first flow through the process I'm getting:
System Error 1219 has occurred.
Multiple connections to a server or shared resource by the same user,
using more than one user name, are not allowed. Disconnect all
previous connections to the server or shared resource and try again.
This happens even when the enumeration says there are no current connections.
My question is: why? Why am I getting this error? Is the authenticated connection to the server still cached? Can I enumerate these authentication tokens? Kerberos? LSA?
I haven't been able to find the smallest foothold of information to progress forward on this project. Any help is appreciated!
I'm trying to remember the solution we used when we came across this problem for a network backup program a few years ago.
I'm certain the solution involves using either WNetAddConnection2 or WNetAddConnection3 instead of WNetUseConnection. I think that passing the flag CONNECT_CRED_RESET should take care of this, but I'm not absolutely certain.
Note that CONNECT_CRED_RESET is only documented for WNetAddConnection2 and not WNetAddConnection3, though MSDN says the only difference between the two is the hWnd parameter for owner of dialog windows - I'd try with WNetAddConnection2 and only if it works, experiment with WNetAddConnection3. You may even get it to work with WNetUseConnection!
Make sure to note the dependencies CONNECT_CRED_RESET has on other flags.

Is an SPN required when using Kerberos with DCOM?

I'm using DCOM to provide various application services on a Windows network, using Kerberos to handle authentication. The system normally works fine, but I'm running into issues accessing the service from a separate (trusted) domain. Particularly, the service is unable to make callbacks to the client application, receiving the error "A security package specific error occurred". Also, if I tweak the service to specifically require Kerberos authentication (rather than using SNEGO/negotiate), the client can't even call into the server (again receiving "A security package specific error occurred").
The confusing thing is that things have been working for years without issue. However, a few things are different here, as compared to what we've done before. For one, the servers involved are running Windows 2008, which I have not previously used. Also, as noted above, the errors are only occurring when the service is accessed from an account from a separate domain, and previous usage has never attempted this.
Now to the question: I'm not using an SPN (service principal name) for this DCOM service, but some of the errors and event logs makes me think that might be the problem. However, all the docs I've found are unclear on whether this is correct, or how I would set up the SPN if I do need it. Does anybody know for sure whether an SPN is what I'm missing here? If so, can you point me to how this should be done?
Additional details:
For the scenario where the server is set to force Kerberos authentication, turning on Extended RPC Debugging gives some additional clues. The client can successfully connect using CoCreateInstanceEx, but calls on the service interface fail as noted above. The RPC error records show an error at location 140, and the error code is 0x80090303 ("The specified target is unknown or unreachable"), and the third parameter for that record is an empty string. This points to a missing SPN as the culprit.
For what its worth, Kerberos by definition requires SPNs. You might be able to utilize the built-in SPNS (host/) and the various flavors that host implies (This alias translation is stored in the AD, but for the life of me, I can't seem to find the article listing where this is found).
I'd start by looking at the cross-domain authentication - It can be tricky with kerberos. I'd first turn on full kerberos logging on both the client and server and see if that turns up anything.
Edit: It looks like I may be somewhat mistaken about this. At least one website I found states that DCOM handles SPNs automatically for you (see the bottom of the page), and I confirmed that clients can successfully connect if they demand Kerberos authentication and use "host/[computername]" as the SPN.
It does look like an SPN is required for the service, if you explicitly require Kerberos authentication when calling CoInitializeSecurity in the DCOM server process. For me, the call looked like this:
Warning: Please do not copy this code directly without ensuring that the values are proper for your security needs.
SOLE_AUTHENTICATION_SERVICE sas;
sas.dwAuthnSvc = RPC_C_AUTHN_GSS_KERBEROS;
sas.dwAuthzSvc = RPC_C_AUTHZ_NONE;
sas.pPrincipalName = L"myservice/mymachine";
sas.hr = S_OK;
CoInitializeSecurity( 0, 1, &sas, 0, RPC_C_AUTHN_LEVEL_DEFAULT,
RPC_C_IMP_LEVEL_DEFAULT, 0, EOAC_NONE, 0 );
The SPN can be configured using setspn, as demonstrated below:
setspn -A myservice/mymachine serviceusername
(see the setspn docs for details).
Unfortunately this still didn't solve my problem, but I think the remaining issue is related to some specific problem with the test machines.
I have come across this error while trying to create an instance of the remote COM object. We may face this error If "Delegate" impersonation level is used by the client during the call to CoInitializeSecurity(), and the COM+ service is running under a user account which doesn't have "delegation" permission at the domain level.

Error 1053: the service did not respond to the start or control request in a timely fashion

I have recently inherited a couple of applications that run as windows services, and I am having problems providing a gui (accessible from a context menu in system tray) with both of them.
The reason why we need a gui for a windows service is in order to be able to re-configure the behaviour of the windows service(s) without resorting to stopping/re-starting.
My code works fine in debug mode, and I get the context menu come up, and everything behaves correctly etc.
When I install the service via "installutil" using a named account (i.e., not Local System Account), the service runs fine, but doesn't display the icon in the system tray (I know this is normal behavior because I don't have the "interact with desktop" option).
Here is the problem though - when I choose the "LocalSystemAccount" option, and check the "interact with desktop" option, the service takes AGES to start up for no obvious reason, and I just keep getting
Could not start the ... service on Local Computer.
Error 1053: the service did not respond to the start or control request in a timely fashion.
Incidentally, I increased the windows service timeout from the default 30 seconds to 2 minutes via a registry hack (see http://support.microsoft.com/kb/824344, search for TimeoutPeriod in section 3), however the service start up still times out.
My first question is - why might the "Local System Account" login takes SOOOOO MUCH LONGER than when the service logs in with the non-LocalSystemAccount, causing the windows service time-out? what's could the difference be between these two to cause such different behavior at start up?
Secondly - taking a step back, all I'm trying to achieve, is simply a windows service that provides a gui for configuration - I'd be quite happy to run using the non-Local System Account (with named user/pwd), if I could get the service to interact with the desktop (that is, have a context menu available from the system tray). Is this possible, and if so how?
Any pointers to the above questions would be appreciated!
After fighting this message for days, a friend told me that you MUST use the Release build. When I InstallUtil the Debug build, it gives this message. The Release build Starts fine.
If you continue down the road of trying to make your service interact with the user's desktop directly, you'll lose: even under the best of circumstances (i.e. "before Vista"), this is extremely tricky.
Windows internally manages several window stations, each with their own desktop. The window station assigned to services running under a given account is completely different from the window station of the logged-on interactive user. Cross-window station access has always been frowned upon, as it's a security risk, but whereas previous Windows versions allowed some exceptions, these have been mostly eliminated in Vista and later operating systems.
The most likely reason your service is hanging on startup, is because it's trying to interact with a nonexistent desktop (or assumes Explorer is running inside the system user session, which also isn't the case), or waiting for input from an invisible desktop.
The only reliable fix for these issues is to eliminate all UI code from your service, and move it to a separate executable that runs inside the interactive user session (the executable can be started using the global Startup group, for example).
Communication between your UI code and your service can be implemented using any RPC mechanism: Named Pipes work particularly well for this purpose. If your communications needs are minimal, using application-defined Service Control Manager commands might also do the trick.
It will take some effort to achieve this separation between UI and service code: however, it's the only way to make things work reliably, and will serve you well in the future.
ADDENDUM, April 2010: Since this question remains pretty popular, here's a way to fix another common scenario that causes "service did not respond..." errors, involving .NET services that don't attempt any funny stuff like interacting with the desktop, but do use Authenticode signed assemblies: disable the verification of the Authenticode signature at load time in order to create Publisher evidence, by adding the following elements to your .exe.config file:
<configuration>
<runtime>
<generatePublisherEvidence enabled="false"/>
</runtime>
</configuration>
Publisher evidence is a little-used Code Access Security (CAS) feature: only in the unlikely event that your service actually relies on the PublisherMembershipCondition will disabling it cause issues. In all other cases, it will make the permanent or intermittent startup failures go away, by no longer requiring the runtime to do expensive certificate checks (including revocation list lookups).
I faced this problem because of a missing framework on the box running my service. The box had .NET 4.0 and the service was written on top of .NET 4.5.
I installed the following download on the box, restarted, and the service started up fine:
http://www.microsoft.com/en-us/download/details.aspx?id=30653
To debug the startup of your service, add the following to the top of the OnStart() method of your service:
while(!System.Diagnostics.Debugger.IsAttached) Thread.Sleep(100);
This will stall the service until you manually attach the Visual Studio Debugger using Debug -> Attach to Process...
Note: In general, if you need a user to interact with your service, it is better to split the GUI components into a separate Windows application that runs when the user logs in. You then use something like named pipes or some other form of IPC to establish communication between the GUI app and your service. This is in fact the only way that this is possible in Windows Vista.
In service class within OnStart method don't do huge operation, OS expect short amount of time to run service, run your method using thread start:
protected override void OnStart(string[] args)
{
Thread t = new Thead(new ThreadStart(MethodName)); // e.g.
t.Start();
}
I'm shooting blind here, but I've very often found that long delays in service startups are directly or indirectly caused by network function timeouts, often when attemting to contact a domain controller when looking up account SIDs - which happens very often indirectly via GetMachineAccountSid() whether you realize it or not, since that function is called by the RPC subsystem.
For an example on how to debug in such situations, see The Case of the Process Startup Delays on Mark Russinovich's blog.
If you are using Debug code as below in your service the problem may arise.
#if(!DEBUG)
ServiceBase[] ServicesToRun;
ServicesToRun = new ServiceBase[]
{
new EmailService()
};
ServiceBase.Run(ServicesToRun);
#else
//direct call function what you need to run
#endif
To fix this, while you build your windows service remove #if condition because it didn't work as it is.
Please use argument for debug mode instead as below.
if (args != null && args.Length > 0)
{
_isDebug = args[0].ToLower().Contains("debug");
}
In my case the problem was missing version of .net framework.
My service used
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
</startup>
But .net Framework version of server was 4, so by changing 4.5 to 4 the problem fixed:
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0" />
</startup>
Copy the release DLL or get the dll from release mode rather than Debug mode and paste it to installation folder,,it should work
I was running into a similar problem with a Service I was writing. It worked fine then one day I started getting the timeout on Start errors. It happened in one &/or both Release and Debug depending on what was going on. I had instantiated an EventLogger from System.Diagnostics, but whatever error I was seeing must have been happening before the Logger was able to write...
If you are not aware of where to look up the EventLogs, in VS you can go to your machine under the Server Explorer. I started poking around in some of the other EventLogs besides those for my Service. Under Application - .NETRuntime I found the Error logs pertinent to the error on startup. Basically, there were some exceptions in my service's constructor (one turned out to be an exception in the EventLog instance setup - which explained why I could not see any logs in my Service EventLog). On a previous build apparently there had been other errors (which had caused me to make the changes leading to the error in the EventLog set up).
Long story short - the reason for the timeout may be due to various exceptions/errors, but using the Runtime EventLogs may just help you figure out what is going on (especially in the instances where one build works but another doesn't).
Hope this helps!
Install the debug build of the service and attach the debugger to the service to see what's happening.
I want to echo mdb's comments here. Don't go this path. Your service is not supposed to have a UI... "No user interaction" is like the definining feature of a service.
If you need to configure your service, write another application that edits the same configuration that the service reads on startup. But make it a distinct tool -- when you want to start the service, you start the service. When you want to configure it, you run the configuration tool.
Now, if you need realtime monitoring of the service, then that's a little trickier (and certainly something I've wished for with services). Now you're talking about having to use interprocess communications and other headaches.
Worst of all, if you need user interaction, then you have a real disconnect here, because services don't interact with the user.
In your shoes I would step back and ask why does this need to be a service? And why does it need user interaction?
These two requirements are pretty incompatible, and that should raise alarms.
I had this problem and it drove me nuts for two days…
If your problem similar to mine:
I have settings “User settings” in my windows service, so the service can do self-maintenance, without stopping and starting the service. Well, the problem is with the “user settings”, where the config file for these settings is saved in a folder under the user-profile of the user who is running the windows service under the service-exe file version.
This folder for some reason was corrupted. I deleted the folder and service start working back again happily as usual…
I had this problem, it took about a day to fix. For me the problem was that my code skipped the "main content" and effectively ran a couple of lines then finished. And this caused the error for me. It is a C# console application which installs a Windows Service, as soon as it tried to run it with the ServiceController (sc.Run() ) then it would give this error for me.
After I fixed the code to go to the main content, it would run the intended code:
ServiceBase.Run(new ServiceHost());
Then it stopped showing up.
As lots of people have already said, the error could be anything, and the solutions people provide may or may not solve it. If they don't solve it (like the Release instead of Debug, adding generatePublisherEvidence=false into your config, etc), then chances are that the problem is with your own code.
Try and get your code to run without using sc.Run() (i.e. make the code run that sc.Run() would have executed).
This problem usually occurs when there is some reference missing on your assembly and usually the binding fails at the run time.
to debug put Thread.Sleep(1000) in the main(). and put a break point in the next line of execution.
Then start the process and attach the debugger to the process while it is starting. Press f5 after it hit the break point. It will throw the exception of missing assembly or reference.
Hopefully this will resolve this error.
Once try to run your exe file. I had the same problem, but when I ran it direct by double click on the exe file, I got a message about .Net framework version, because I was released the service project with a framework which it wasn't installed on target machine.
Took me hours, should have seen the event viewer get_AppSettings().
A change in the app config, caused the problem.
Adding 127.0.0.1 crl.microsoft.com to the "Hosts" file solved our issue.
My issue was due to target framework mentioned in windows service config was
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.6"/>
</startup>
and my server in which I tried to install windows service was not supported for this .Net version.
Changing which , I could able to resolve the issue.
I had a similar issue, steps I followed:
Put a Debugger.Launch() in the windows service constructor
Followed step by step to see where it got stuck
My issue wasn't due to any error.
I had a BlockingCollection.GetConsumingEnumerable() in the way that caused the windows service to wait.
I had this problem too. I made it to work by changing Log On account to Local System Account. In my project I had it setup to run as Local Service account. So when I installed it, by default it was using Local Service. I'm using .net 2.0 and VS 2005. So installing .net 1.1 SP1 wouldn't have helped.
Both Local System Account and Local Service would not work for me, i then set it to Network Service and this worked fine.
In my case, I had this trouble due to a genuine error. Before the service constructor is called, one static constructor of member variable was failing:
private static OracleCommand cmd;
static SchedTasks()
{
try
{
cmd = new OracleCommand("select * from change_notification");
}
catch (Exception e)
{
Log(e.Message);
// "The provider is not compatible with the version of Oracle client"
}
}
By adding try-catch block I found the exception was occuring because of wrong oracle version. Installing correct database solved the problem.
I also faced similar problem and found that there was issue loading assembly. I was receiving this error immediately when trying to start the service.
To quickly debug the issue, try to run service executable via command prompt using ProcDump http://technet.microsoft.com/en-us/sysinternals/dd996900. It shall provide sufficient hint about exact error.
http://bytes.com/topic/net/answers/637227-1053-error-trying-start-my-net-windows-service helped me quite a bit.
This worked for me. Basically make sure the Log on user is set to the right one. However it depends how the account infrastructure is set. In my example it's using AD account user credentials.
In start up menu search box search for 'Services'
-In Services find the required service
-right click on and select the Log On tab
-Select 'This account' and enter the required content/credentials
-Ok it and start the service as usual
In case you have a windows form used for testing, ensure that the startup object is still the service and not the windows form
We have Log4Net configured to log to a database table. The table had grown so large that the service was timing out trying to log messages.
open the services window as administrator,Then try to start the service.That worked for me.
Build project in Release Mode.
Copy all Release folder files to source path.
Execute Window service using command prompt window in administrative access.
Never delete files from source path.
At lease this works for me.
Release build did not work for me, however, I looked through my event viewer and Application log and saw that the Windows Service was throwing a security exception when it was trying to create an event log. I fixed this by adding the event source manually with administration access.
I followed this guide from Microsoft:
open registry editor, run --> regedit
Locate the following registry subkey:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Eventlog\Application
Right-click the Application subkey, point to New, and then click Key.
Type event source name used in your windows service for the key name.
Close Registry Editor.

Resources