I have installed OEM 13c and deployed a couple of agents and want to test out the Log File Monitoring utility. I have enabled it and added a log file to monitor.
When I go and test it out, it does not show any alerts when they are put into the Log File. On the agent server, I have tailed the file and see the messages coming into the log file.
Does anyone have experience adding log files to OEM? I could have configured it wrong. Or is there any troubleshooting steps that I can follow to see if the server is even contacting the agent for reading the log file. Status of the agent is good with no incidents.
Without access to the system, it would be difficult to tell you the exact cause of this issue. However, I can list a few potential causes of this issue that I have experienced personally:
Permissions. The Oracle Enterprise Manager Agent is very convoluted when it comes to system permissions within a remote server. The agent can be owned and run as any number of users but during metric evaluation, may also need sudo or pam-authentication permissions to access certain entities on the server. Depending on the authentication profiles on that server, this could be the cause of your issue. There are ways to grant the agent access through the PAM stack if that is necessary.
Syntax. The wildcard syntax in the OEM GUI can be a little confusing as well. I would play with the wildcard elements a bit on the "String" component to ensure that it isn't as simple as adding wildcards to the beginning and end of the string. Without diving into the binaries of the agent plugins, it is difficult to assess exactly how the agent is evaluating this particular metric
One suggestion I would have is to go through the agent commands. There are specific commands you can run to manually force an agent to evaluate a particular metric for a particular target. This can allow you to manually trigger the metric collection locally on the server and evaluate what exactly is being performed at the agent level.
On the system I was running (12c) the command was as follows:
emctl control agent runCollection <hostname>:host host_storage
Related
I am running a windows 2016 server, we are running IIs 10 on it and i need to be able to assert if there is an AppPool setup before i deploy a website. If it doesn't exist i need setup the AppPool with a specific user and password.
All of this is done using a release agent through Azure Devops.
The agent is running as a NON-ADMIN, and i all accounts involved are running as NON-ADMIN. I have no intention at all to run any admin accounts, for security reasons i want to give least privildges to all accounts involved.
when i try to set up a AppPool using appcmd.exe i get the error msg:
KeySet does not exist.
When running everything as admin it works (and i have absolutely no intention in running any of this as admin).
What i have tried:
i have added the non-admin account to the IIS_IUSRS group.
Made sure that the user has read permissions to the file: 76944fb33636aeddb9590521c2e8815a_GUID in the %ALLUSERSPROFILE%\Microsoft\Crypto\RSA\MachineKeys folder.
i have tried everything here: Error when you change the identity of an application pool by using IIS Manager from a remote computer
anyone that actually knows the cause of this problem?
UPDATE:
Microsoft clearly recommends that agents should be run using service accounts, which i am doing and i have no interest in giving build agents administrative rights to 1000s of servers when they clearly don't need that kind of powers actually. I want to restrict their powers to only be allowed to do what they need to do. I can't believe that giving everything admin is apparently the norm.
After a lot of googling, and i mean A LOT. I managed to solve this. And let me say, that it baffles me that "least privileged accounts" is not common practice in the Microsoft and windows world.
I found this excellent post by InfoSecMike locking down azure devops pipelines.
And we both have the exact same requirements and opinions on this topic.
You CLEARLY don't need admin rights to update IIs configurations (because that would be insane, right!?). The IIs configuration API does not care what rights you have, what you do need is access to certain files. But this is not documented. Microsoft themselves, just for simplicity, tells you that you need to be admin, and buries all the details really deep in documentation when this should be best practice. Also what amazes me is that no one questions it.
What you need is the following:
full access to C:\Windows\System32\inetsrv\Config
full acccess to C:\inetpub
read access to three keys in C:\ProgramData\Microsoft\Crypto\RSA\MachineKeys\
6de9cb26d2b98c01ec4e9e8b34824aa2_GUID (iisConfigurationKey)
d6d986f09a1ee04e24c949879fdb506c_GUID (NetFrameworkConfigurationKey)
76944fb33636aeddb9590521c2e8815a_GUID (iisWasKey)
The 2 first bullet points can be obtained if you make sure your service account is a member of the group IIS_IUSRS.
This group will not give you access to the keys. You need to manually give read permissions to these 3 keys to the agent user.
If you don't give access to these keys you will get the obscure error message
Keyset does not exist ( exception from HRESULT : 0x8009000D)
Which is an incorrect error if you ask me as it should be an IllegalAccessException with proper reason telling you that you don't have access to read the key because the keys are there, they do exist (nice code microsoft, maybe you should open source this so we can fix).
I'll leave with this quote from infosecmike.
The goal was to lock down the permissions of the Azure Pipeline Agent {...}. I started Googling, pretty sure I would find a way to achieve this goal. I didn’t. It’s surprising to not find an answer about this. It seems like the principle of least privilege does not apply anymore in a devops world.
This is why i prefer Linux over Windows. This is a simple task there.
I'm installing the latest MMASetup-AMD64.exe and want to hook up to Log Analytics AND SCOM. But, I'm having trouble finding the command line parameters for SCOM. Does anybody know them? The Log Analytics ones are well documented and are here:
ADD_OPINSIGHTS_WORKSPACE=1
OPINSIGHTS_WORKSPACE_ID="1234"
OPINSIGHTS_WORKSPACE_KEY="5678"
I need the equivalent parameters for management group name and management server. Effectively completing these boxes but via the command line.
Thanks in advance.
I believe parameters for management group, secure port, etc. are not available with MMASetup-AMD64.exe. Here are the supported command line options with it. So may be, if its feasible in your environment and setup, try to use MOMAgent.msi to install agent manually or to deploy System Center Operations Manager agents from the command line or by using the Setup Wizard. The parameters like MANAGEMENT_GROUP, SECURE_PORT, etc. are all explained along with examples in this document. For more information, please refer it.
Other references related to OM agents and OM groups:
Process manual agent installations
Configuring Windows agents
Operations Manager agents
Creating and managing groups
Connecting management groups in Operations Manager
Planning a Management Group Design
I am responsible for running centralized backups of Windows Security logs on a network of ~15 Windows boxes. To automate this task, I have been writing a Powershell script that utilizes wevtutil's /r parameter to do it all remotely. All of the boxes are connected to a Sharepoint network drive that I was hoping to copy the logs to so that I could centralize all of the logs, but I've run into some trouble.
The script runs fine when I pass it the ip of the Windows box that it's running on. The logs are copied to the Sharepoint without a hitch. The script also runs fine when I just tell it to copy the logs locally. However, when the script trys to remotely copy the computer's log to the sharepoint, I get a Failed to Archive Security log. The account is not authorized to log in from this station. error.
The format of the command is
wevtutil epl Security \\path\to\sharepoint\[hostname]-[datetime]Security.evtx /r:[hostname]
I am running the script as a domain admin. I have also ran the script with the credentials of a local admin and I got a generic access denied error.
A google search for the error message mostly includes computers not being able to access network drives (not remote wevtutil specifically) and include a lot of messing around in
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters
or
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters
which have not yielded any results. I am also restricted to Powershell 1, so I cannot use Powershell itself remotely AFAIK.
I could just put the script on each machine and run it locally with task scheduler, but I was hoping for a more elegant solution. Does anyone have experience with using wevtutil in this way and can point me in the right direction, or perhaps even suggest a better technique/tool?
How much latitude do you have to implement another solution?
If you wanted to do something with the logs, or make them easily searchable, you could set up a free Splunk server and either use a Splunk forwarder to ship the logs off box, or you could also use Powershell to send the logs to Splunk's HTTP event collector.
The scenario is as follows:
I have TeamCity set up to use AWS EC2 hosts running Windows Server 2012 R2 as build agents. In this configuration, the TeamCity agent service is running as SYSTEM. I am trying to implement FastBuild as our new compilation process. In order to use the distributed compilation functionality of FastBuild, the build agent host needs to have access to a shared network folder. Unfortunately, I cannot seem to give this kind of access from one machine to another.
To help further the explanation, I'll use named examples. The networked folder, C:\Shared-Folder, lives on a host named Central-Host. The build agent lives on Builder-Host. Everything is running Windows Server 2012 R2 on EC2 hosts that are fully network permissive to each other via AWS security groups. What I need is to share a directory from Central-Host so that Builder-Host can fully access it via a directory structure like this:
\\Central-Host\Shared-Folder
By RDPing into both hosts using the default Administrator account, I can very easily set up the network sharing and browse (while on Builder-Host) to the \\Central-Host\Shared-Folder location. I can also open up the command line and run:
type NUL > \\Central-Host\Shared-Folder\Empty.txt
with the result of an empty text file being created at that networked location.
The problem arises from the SYSTEM account. When I grab PSTOOLS and use the command:
PSEXEC -i -s cmd.exe
I can test commands that will be given by TeamCity. Again, it is a service being run as SYSTEM which, I need to emphasize, cannot be changed to a normal User due to other issues we have when using TeamCity agents under the User account type.
After much searching I have discovered how to set up Active Directory services so that I can add Users and Computers from the domain but after doing so, I still face access denied errors. I am probably missing something important and I hope someone here can help. I believe this problem will be considered "solved" when I can successfully run the "type NUL" command shown above.
This is not an answer for the permissions issue, but rather a way to avoid it. (Wanted to add this as a comment, but StackOverflow won't let me - weird.)
The shared network drive is used only for the remote worker discovery. If you have a fixed list of workers, instead of using the worker discovery, you can specify them explicitly in your config file as follows:
Settings
{
.Workers =
{
'hostname1' // specify hostname
'hostname2'
'192.168.0.10' // or ip
}
... // the other stuff that goes here
This functionality is not documented, as to-date all users have wanted the automatic worker discovery. It is fine to use however, and if it is indeed useful, it can be elevated to a supported feature with just a documentation update.
I always forget to check what's going on in IIS on our webservers, and am wondering: is there some stupid applet or something that always runs locally that I can click on to check event logs and IIS logs on a remote machine?
Mark
You can set up samurize to follow the output of the logging on the local and remote machines but it requires some setup.
You can use a remote shell utility such as OpenSSH to connect to remote machines securely.
One at a time. Compmgmt.msc -> connect to another computer.
But one at a time is boring. Monitoring dozens of machines? I've been using logparser from MS for my log monitoring needs. I run a query that dumps errors and warnings to a csv file a few times a day.
So far, I've only used it to aggregate event logs across the dozen servers in our QA environment, but it appears to take many forms on log input, including IIS. A pseudo log file query (don't have samples with me)
logparser "Select [eventtype], [message] into output.csv FROM \\server1\system, \\server2\system" -i EVT
This shows: You can aggregate multiple servers. You tell it the input format - it supports a dozen log types. You can dump it into a csv file. It looks sort of like SQL. This article in security focus has an IIS log sample.
I'm not an applet type of guy, so I haven't though much about desktop widgets to do this.