Basic Monitoring and Automation

Article Series

<< Prev - Mixed OS Business Environments - Next >>

This post is part of a series that is focussing on setting up a standardized IT environment in a mixed Operating System setting. In this post I will be focussing on the basics of configuring Basic Monitoring and Automation services for the purpose of maintaining a standardized environment.

Mixed OS Business Environments

This post is part of a series that is focussing on setting up a standardized IT environment in a mixed Operating System setting. It is important to be able to efficiently manage any IT environment. In very small businesses it can be possible to only run OS’s from the same family (Microsoft, Linux, Apple), but the large a business becomes the more likely they are to need a mixed environment to meet business needs. Employees working in a cubicle on basic tasks can easily all use the same version of Windows, but if the company needs to run application or database servers they will eventually want to use Linux for many of those purposes.

In this post I will be focussing on the basics of configuring Basic Monitoring and Automation services for the purpose of maintaining a standardized environment. To do this I have made use of AVG Managed Workplace, a Windows 2012 R2 Server, a Windows 10 Desktop, and a Fedora 25 Linux Server. I’m using AVG Managed Workplace to configure monitoring and automation policies, which requires WMI on Windows Computers, and SSH and SNMP on Linux computers.

AVG Managed Workplace Configuration

Getting started with Managed Workplace (MW) is fairly straight forward should you choose to use it. I’ll skip over things like pricing, suffice it to say this is not a free service, and just discuss the technical capabilities and basic procedure for configuring it for this purpose. MW works by first creating a Site. A Site is intended to be used to manage the IT environment for a single physical location, however it also capable of managing additional machines remotely. Configuration is done through AVG’s online Service Center portal. The basics of configuring a site during creation includes selecting a service plan (the services that will automate IT maintenance and monitoring), setting up secure credentials, configuring the network scanner settings, and installing the OnSite Manager.

Service Plans

Service plans in MW are used to add configured services to your IT environment. These can include things like Windows Patch Management, Antivirus Software Deployment, Standardized Software Installations, Network and Device Monitoring, and many other applications. For the purposes of this article I’ll be focussing on the services I’ve specifically named.

The services themselves can be customized significantly. In the case of Patch Management, updates can be manually allowed or policies can allow updates and upgrades based on criteria such as being a critical OS or security update. MW being AVG means that it comes with AVG Antivirus deployment and configuration tools. Other software installations are typically managed by setting up an automation policy looking for the absence of a particular software package combined with a policy to push an installation script to devices in the group. Network and Device Monitoring is accomplished through services with configured monitoring policies.

Many monitoring tasks have preconfigured policies available for immediate use to monitor systems through WMI or SNMP. By using these preconfigured monitoring policies devices that have either WMI or SNMP enabled can be monitored and their hardware and OS status reported, as well as monitoring things such as resource use over time.

Secure Credential Setup

MW makes use of a single OnSite Manager that in most cases does not require an agent to be present on managed devices and systems. This is accomplished by providing the Site with credential sets that have privileges on the managed systems. For windows systems that have been added to a domain, you can simply create a Domain Administrator account named MWservice and all devices with domain user authentication will be manageable so long as they have WMI enabled. For devices and systems that are either on a workgroup, or do not support WMI a local administrator account will need to be added, and in the case of non-WMI devices SSH and SNMP will also need to be configured.

Network Scan Settings

The network scan settings configures OnSite Manager’s network probe. The network probe is used to automatically discover devices on the network so they can be added or excluded from MW management. The configuration settings available consists of being able to add IP ranges or device names to search for, and additional policies can determine if discovered devices are either auto included or excluded from being managed by that OnSite Manager (OM). Alternatively, for Windows and OSX there is an option to install device managers to add those devices to the site for management by the OM. Devices with a Device Manager installed do not need to be located on the same physical network, making this a useful option for laptops that may be taken off-site.

OnSite Manager Installation

The OnSite Manager may be installed on either a 64-bit Windows Workstation OS running a currently supported OS with hardware that meets minimum technical specifications, or a currently supported Windows Server OS that meets the minimum technical specifications. This installation requires that the 3.5 and 4.5 .Net Frameworks be installed, and will also install and configure a Microsoft SQL Server Express instance. I’ve installed on a Windows 2012 R2 Server, the minimum specifications for my purposes required at least a dual core processor, 4 GB of ram, and about 60 GB of space for the OM the SQL Server and a local Patch Management distribution service. The installation needs to be performed using a Domain Administrator account for Active Directory Domain environments, or a local admin account for a work group.

Network and Device Monitoring and Task Automation

In MW this is managed through the use of Monitoring policies, and requires that either WMI or SNMP be enabled for the monitored devices unless they have a device manager installed. Because device managers are only available for Windows, OSX, Android, and iOS; devices running Linux and Network Devices such as switches, routers and printers will need SNMP configured for monitoring purposes.

Automated tasks as mentioned previously is managed through the use of automation policies, and requires either WMI or SSH access to the device. There are many different types of network devices such as routers and switches which require specific setup for both SNMP and SSH, and not all such devices support SNMP or SSH. In a later article I will be discussing Network Administration, and how to configure necessary services in that context, for this article I will be focussing on how to configure SNMP and SSH. I would include instructions for WMI, however MW provides a free setup tool for Windows devices that configures and enables WMI for the user.

Linux Systems Configuration

In most enterprise environments two flavors of Linux/Unix are likely to be used. For workstations Ubuntu Linux is typically used, and for most Server installations CentOS is most commonly used. These two flavors of linux/unix do have many things in common, but there are a few key differences that should be noted while configuring SSH and SNMP services. In Ubuntu, package management, software installations, and library dependency resolution is handled through the use of either the Ubuntu app store, or with the “apt-get” command in the terminal. For RedHat Unix varieties (CentOS or Fedora), the “yum” or “dnf” commands are used. There are subtle differences between these commands and reading the manual for whichever version you’re using is recommended so you can understand the different parameters used for each. For this article I’ll only be providing examples using “dnf” and other commands available to RedHat Unix flavors.

Before proceeding with configuring SSH and SNMP, basic system configuration should be done first. A local account with sudo privileges should be added to the linux machine. It is possible to add a Linux system to an Active Directory domain environment, as well as configure AD Group based authentication, but I will be discussing that topic in a future article so I will assume a local sudo user for this tutorial. It is recommended that this sudo enabled account not be used by any actual users, I personally prefer to create an account named “mwservice” for this purpose.

After an appropriate administrator account has been created you will also need to configure the Linux firewall to allow in-bound SSH and SNMP traffic to make connections. This is done either through ip-table configuration or through GUI based configuration tools. It is recommended that only SSH and SNMP traffic from IP ranges limited only to computers of administrators and the OnSite Manager be allowed through. For SNMP, if the firewall does not block in-bound traffic, IP based connection rules may also be configured directly in the SNMP service.

Configuring SSH

For RedHat based Linux systems the SSH daemon comes pre-installed, and configuration is very simple. First, it is recommended that you disable direct root login privileges over SSH. This is done by editing the SSH configuration file “/etc/ssh/sshd_config” look for the line that says “PermitRootLogin yes” and change the “yes” to “no”. The PermitRootLogin line may be commented out with “#” in which case remove the “#.” If SSH is already running you’ll need to restart the SSHD service after making this change. It is also possible to limit SSH login to a specific user group or a list of users with the “AllowUsers” directive and proper Domain Authentication configuration.

Starting the SSH service daemon and setting it to always start on system startup is easily accomplished with two lines so long as the SSH utility has not been previously uninstalled.

#systemctl start sshd
#systemctl enable sshd

Please note these are terminal commands, and the “#” symbol indicates this should be executed from a terminal shell with root permissions. To temporarily stop SSH run the first command, but replace start with stop. If you no longer want SSH to start on system startup run the second command replacing enable with disable. Additionally, if you are experiencing trouble with the SSH daemon, run the “#systemctl status sshd” command to view the most recent system log entries relevant to the sshd service.

The easiest way to verify if you have successfully setup SSH is to use PuTTY to attempt to make an SSH connection from the Windows computer running the OnSite Manager. I always verify if several users that should have access can log in via SSH, and that users that should not have access cannot. It’s also possible to limit the number of simultaneous SSH connections, which can be tested by opening multiple PuTTY sessions. Additionally, if you have configured remote services to only be accessible over SSH you can also use PuTTY to open an SSH session with a service tunnel to verify they work properly as well.

SNMP

In both Ubuntu and RedHat Linux the commonly used SNMP service daemon is net-snmp. This SNMP tool is also available for use in Windows with some configuration. It provides the ability to set up an SNMP service, as well as client applications to make SNMP queries. This makes it useful for testing SNMP services to verify they work. There are multiple net-snmp modules that need to be installed for the service to have full functionality, and I’ll provide the terminal commands you should run below:

#dnf -y install net-snmp net-snmp-libs net-snmp-utils

This will install the basic snmp client and daemon service in addition to necessary libraries for SNMPtraps, some automated configuration tools, and the standard set of MiB files for SNMP versions 1, 2, and 3. When configuring SNMP you will want to decide which version of SNMP you intend to use. Versions 1 and 2 are similar with minor difference, but both lack transport layer security measures and do not use any form of authentication. They limit access to any request that knows the correct “community” string, which can be easily captured by applications like wireshark passively listening to network traffic. For this reason I recommend using SNMP version 3, which has user based authentication and connection encryption options. I will provide examples of how to configure both Version 1/2 and Version 3 SNMP services.

For Version 1/2 SNMP it is possible to create a very simple configuration file that will grant access to anyone that knows the default community string is “public”, however this is extremely insecure and could potentially allow for attacks. For this reason I recommend not using the default community string, and setting up group based permissions directives. To do this you’ll need to edit “/etc/snmp/snmpd.conf” with a text editor. I recommend learning and using vim as Linux servers do not typically have GUI’s to make GUI based text editors like gedit an option.

First you want to start by creating security groups. This is done using the “com2sec” directive, which can set the IP range to accept specific community strings from and associates that string with a security name. Security names can then be associated with group names via the “group” directive, which also limits which versions of SNMP may make requests using the associated community strings. Next, you can set the permissions by group name including authorization and encryption requirements, read, write, and notification rights according to the version of SNMP used to connect. You will also need to configure which subtrees are accessible using the view directive, which can also be specified per group name or read only/read write. A basic setup for this is included below:

##              sec.name       source                    community
com2sec   local                localhost               aStringYouMakeUp
com2sec   mynetwork    192.168.0.0/24   aStringYouMakeUp

##         group.name     sec.model   sec.name
group   MyRWGroup   v1                  local
group   MyRWGroup   v2c                local
group   MyROGroup   v1                   local
group   MyROGroup   v2c                local

##                                   context   sec.model   sec.level     prefix   read   write   notif
access MyROGroup    “”             usm              authpriv    exact     all      none   none
access MyRWGroup   “”             usm              authpriv    exact     all    all    all

##               incl/excl   subtree                          mask
view   all    included   .1                                      80

In addition to these settings, for both SNMP versions 1/2 and 3 you will want to set if you’re logging SNMP connections and both the system location and system administrator contact details as shown below:

syslocation   A concise description of the physical/virtual location of the system
syscontact   Administrator’s first and last name <[email protected]>

dontLogTCPWrappersConnects no

As previously mentioned, SNMP V 1/2 does not use encrypted communication or require user authentication, both of which are supported by SNMP V3. Configuring to use SNMP V3 is done somewhat differently, as the com2sec and group directives don’t configure or restrict access for version 3. To configure a version 3 user you’ll first need to use terminal commands as shown below:

#net-snmp-create-v3-user -A userPassword -a passwordHashType -X communicationEncryptionKey -x communicationEncryptionType userName

This command will add the user credentials to the snmp user configuration file, and a single line entry to the /etc/snmp/snmpd.conf file giving read/write privileges to the user. This basic configuration is reasonably secure, but won’t configure the SNMP agent to listen for connections from a limited ip range or configure which views the user has access to by default. For this reason I recommend editing the config file to be similar to below:

rwuser   userName  authpriv .1
agentaddress 192.168.0.1:161
agentaddress 127.0.0.1:161

These directives first give userName read/write access privileges, requires them to use both user authentication and communication encryption, and last sets the view the user has access to. The agentaddress directive sets the ip address the SNMP agent will accept connections from (default is localhost and the system’s ip), and can be set for udp/tcp/both and set the local port the agent listens on(port 161 is the default). Once you’ve configured SNMP you’ll need to start and enable the snmpd service similar to how you started and enabled the sshd service. Keep in mind that systemctl status sshd is a good command to run after starting snmp to verify if there were any errors in the configuration file and that no other issues occurred. Just a reminder from earlier, the firewall needs to be configured to allow snmp traffic in addition to the steps above.

To test your configuration there are a number of MiB browsers available for free that can test SNMP Versions 1/2. For testing SNMP version 3, the simplest solution is to use the snmpwalk command from another linux machine, or in Windows you can either enable Windows experimental linux bash support or setup net-snmp in you windows environment. MiB Browser use is different for each browser, but if you choose to use net-snmp to test your connection the following terminal commands can test your connections

For versions 1/2
#snmpwalk -v1/v2c -c aStringYouMakeUp ipAddress/localhost system
For version 3
#snmpwalk -u userName -A userPassword -a passwordHashType -X communicationEncryptionKey -x communicationEncryptionType -l authPriv ipAddress/localhost -v3 system

In both commands you can substitute system for another view name or OID number.

On a few last notes about SNMP, this service is capable of setting up for even higher security settings than described here. It is possible to configure SNMP to require user authentication, communication encryption, and additional protections including requiring all snmp connections be made via tls or ssh tunnel. However, Managed Workplace does not have any easy way to be configured to require tls or ssh connections in combination with SNMP. For that reason I won’t be describing these more advanced setup procedures in depth. If you wish to use them you’ll first need to make sure the tls service you wish to use if properly installed and configured, including setting up modules to redirect tls to snmp after packet decryption. Setting SSH for this purpose requires similar configuration. If you have configured everything correctly, the only additional step required is to add the SNMP connection credentials to your MW site, and Linux devices will begin reporting monitoring data via SNMP and you’ll be able to create automation policies to standardize software installations as well as update scheduling through SSH.


References

Fedora 25 System Administrators Guide https://docs.fedoraproject.org/en-US/Fedora/25/html/System_Administrators_Guide/index.html
net-snmp http://net-snmp.sourceforge.net/
AVG Managed Workplace http://www.avg.com/us-en/managed-workplace