Leveraging Small Computers for Enterprise Applications

Infrastructure Project for Direct Financial Solutions[5] – Completed 2016

How Can Small Computers Meet Enterprise Needs?

When people discuss Enterprise needs and challenges they often think about the most expensive equipment or processes and applications that are difficult to develop. For accounting, the costs and challenges of implementing SAP[1] or similar infrastructure projects are often the focus. For marketing, the difficulties of collecting and analyzing marketing data is extremely important. In IT, the costs and challenges of owning and maintaining Servers and in house developed applications is the center of discussion.

What people don’t often discuss, is how information is distributed. Most people assume that simply using email, or a content distribution system is the full answer. While email or a server that manages access to reports or other information can solve most needs, that doesn’t address what the user views that information with. Reports on last month’s sales may only be necessary for a few people to read before a meeting, but there is data that is either being updated in real time or is being displayed for everyone in the room such as a presentation.

For example, on a call floor, supervisors and team leads often need to get up from their desk and assist others. They also need to be able to keep an eye on information like the number of callers on hold, the hourly call completion rate and other information that is updated in real time and useful to everyone on the call floor. In a large IT department with multiple projects being worked on, it can be useful to have a build monitor (a GUI displaying the results of full project compilations and Unit Tests for the current Development branch) commonly displayed. In IT, system and Network administrators commonly need to view their Network status through applications such as Netflow or Nagios Network Analyzer. Help desk employees find it valuable to have a common display of new Help Desk tickets as they come in.

Some companies distribute tablet computers, however this requires that each person have their own or display the data on their work desktop. Both of these solutions involve redundant use of screen space which is not easily shared, and purchasing tablets for a large number of people can become unjustifiably expensive. Other companies use large wall displays and projectors with smart technology that can meet many of these needs, but they are expensive and lack customization options unless they’re displaying a computer’s desktop. Personally, I’ve worked IT System’s Administration, and among my duties were performing cost benefit analysis for computers and displays purchased by the company. Workstation computers and laptops can be very expensive, and often provide far more power than necessary to accomplish these goals.

As a result, it can be far more cost effective to purchase a large display or projector with sufficient resolution that lacks “smart” features, and purchase an inexpensive “small” computer. By a small computer, what I’m referring to are commonly called single board micro-computers. They consist of a circuit board with built in processors, peripheral connector pins or ports such as USB and HDMI, and a minimal quantity of on-board RAM. One of the most common and popular small computers is the Raspberry Pi[2]. A standard Raspberry Pi costs $35, a case and necessary peripherals costs less than $30, putting the full cost of a single Raspberry Pi at less than $70. Considering that the average low-end workstation computer costs $400 dollars, and the difference in price between a high end display with powerful enough “smart” features and similar models without can easily cost in the thousands, this makes the Raspberry Pi a cost effective solution.

The Project Goals for Direct Financial Solutions (DFS)

The purpose of this project was to enable the large displays in the IT department and call floors at DFS to view highly relevant real time information from anywhere in the room without requiring each person to log in and refresh their data individually. On the call floor, they monitored the number of incoming, in-progress, and completed calls along with information including average wait and call completion time. In the IT department displays were needed for a build monitor, as well as task management including a list of pending and in progress development tasks. An additional monitor was also needed for help desk task management, as well as displaying the network monitor information. All monitors were also used for occasional team building and morale purposes, including displaying the results of a company Foosball tournament.

Requirements:

  • The Pi must be configurable to change between display tasks automatically
  • Information being displayed was accessible via webpage
  • Some webpages required secure automated log in management
  • Raspberry Pi setup needed to be automated, including drive imaging and display specific configuration
  • The DFS Network utilized Power Over Ethernet (POE), and it was requested the Pi’s use this as their power source
  • The Pi’s were required to run a full Linux based OS distribution, preferably Fedora
  • The Display needed to be responsive, and display full 1080p
  • Remote login to the Pi’s were required to interface with an Active Directory server for permissions
  • Remote login required access to the Desktop 0 screen
  • Pi’s should turn themselves on, and shut themselves down on a set schedule

Project Challenges

Feature Availability and Performance Concerns

At the time of this project the most advanced Raspberry Pi available was the Pi 2 B, which utilized the ARM Cortex-A7 cpu. This was sufficient to install and run Fedora, but a number of features were not available and Gnome was not sufficiently responsive. These issues were resolved by only installing the Fedora minimal version, and installing the KDE gui, with Thunar and the Xfce 4 terminal environments. By not installing more the minimum necessary applications, greater performance was achievable than typically expected from a standard Raspberry Pi OS.

Configuration Difficulties

A number of terminal configuration commands did not work on the Pi. Specifically, certain pam, winbind, samba and realm configuration commands did not correctly join the local domain or configure Active Directory group access permissions and login settings. This was resolvable by using manual configuration file editing methods. Specifically LDAP and domain server routing options needed to be configured in a number of files.

By default, Raspberry Pi’s do not output large resolutions, such as 1920 x 1080p and Fedora resolution configuration commands cannot change the resolution for this setup. As a result the display initially did not fill the screen, leaving significant unused area. It was possible to increase the screen resolution, but this had to be configured by manually editing configuration files specific to the Raspberry Pi in order to enable and set larger resolutions, rather than being configured through Fedora’s settings.

Network adapter settings, including MAC addresses and IP addresses are stored in the system configuration files rather than stored as persistent by the device, resulting in cloned MAC addresses and IPs on initial imaging. To resolve this type of issue it is possible to simply keep an individual image for each Raspberry Pi. However, this will result in duplication of effort, and if a larger number of such displays are needed (which is very likely for large companies), this is not a maintainable solution. Instead, it is recommended that once a base image containing all configuration needed by all Pi’s is completed the MAC address and IP address at that time is treated as reserved for Pi’s being imaged. These settings will then need to be changed after each new Pi is imaged. This is most easily accomplished by writing a bash script or application and setting it to launch on startup after imaging is completed. I discuss how this is best done in the implementation section.

Raspberry Pi’s also do not natively have the ability to turn themselves on at a scheduled time, and they do not come with a on/restart button built into the design. Pi’s turn themselves on as soon as they are connected to an appropriate power source (micro USB 2/2.5 A). Also, after shutting down, a Raspberry Pi must have its power source disconnected before it can be started again, except in the case of rebooting. Because we were utilizing POE as the Pi’s power source it was possible to configure the routers and switches to turn off the power to their port and then turn it back on following a schedule, enabling the ability to schedule when each Pi starts each day. The ability to shut down at a scheduled time is a configurable option in Fedora, and shutdown times can also be scheduled by startup scripts if more advanced scheduling is needed than provided by the OS system control panel.

Hardware Requirements

Raspberry Pi’s do not natively have the ability to accept POE as their power source, and could potentially damage the Raspberry Pi if setup incorrectly. This was resolved by purchasing a $5 adapter that accepts a POE connection, and splits the power out via a micro-usb port and the Network connection separately. Raspberry Pi’s are powered via micro USB, similar to cell phones and require 2 Amps for the Pi 2 B, and 2.5 for the new Pi 3 B. This allowed us to use our POE connection to both power and connect the Pi to the local network.

Project Implementation

Initial OS and GUI Installation

After careful consideration, and experimenting with the standard Pi OS options (including Raspbian, Ubuntu Mate and a few others), it was decided that the standard options were not sufficiently configurable, easily maintained, or they were not responsive enough. We therefore chose to build and maintain a more custom setup. DFS has frequently used Red Hat Linux distributions, and chose to use Fedora because it offered a minimal install with easy access to all necessary features. Fedora is also free to use, and there is a version of Fedora designed to run on Arm7 Cortex processors. Initial OS installation required following a more complicated procedure than other Raspberry Pi OS’s, which can be found here[3]. After installing the Fedora minimal OS we then configured it to use KDE as its desktop environment. We also configured the terminal to use Xfce 4, and used dnf for dependency package management and application installations.

Something important to understand about Raspberry Pi’s is their primary drive is an SD-card, and installation media such as a CD or USB is not used. Basic installation is performed by using an SD-card writer to format the card and then write the installer to the card which functions as the Pi’s primary hard drive. On starting the Pi after writing the drive image, an OS setup process starts automatically. At the time of this project, this was the only method for installing an OS. On the new Raspberry Pi 3 new boot modes including boot from USB and standard disk drives have been added, but must be manually enabled first.

Fortunately, the use of SD-cards as the primary drive has some benefits for this type of project. 32 GB SD-cards capable of being a Pi’s primary drive are not particularly expensive, and making a backup drive image is often a very simple task requiring only standard hardware and free to use software. This means that once the initial setup is complete, it’s a simple task to make a backup image in the event that something goes wrong later in the configuration process. You’re also able to take backup images before making significant changes, and the drive size means doing so shouldn’t require an unreasonable amount of storage space to keep multiple backup images. Pi drives are also easily swapped out, making it possible to image a new drive and configure it on a spare Pi, then swapping the drive into the desired Pi without taking the Pi’s apart.

Automatic Start/Shutdown and Login Setup

As previously mentioned, Pi’s do not have the ability to turn themselves on at a scheduled time. This is primarily because that requires hardware that is not strictly necessary for a PC to function, and Raspbery Pi’s are designed to be inexpensive while being capable of performing most basic tasks such as web browsing and text editing. Instead, Pi’s are powered on similar to most computers in the mid 90’s and earlier, by power cycling after a full shutdown. Because we were using POE as our power source for the Pi’s we were able to set up procedures to power cycle the Pi’s, both on demand, as well as on a schedule.

Once the Pi’s booted, it was necessary for them to automatically log in to an account which had minimal access and control. Because of the specific quirks of running a minimal install with custom configuration, auto login could not be managed from the system login manager. In order to enable auto login after booting, the configuration files for both Lightdm and Pam were manually edited. Specifically, in Lightdm a Seat Default had to be added and in Lightdm and Pam autologin had to be enabled before setting up  an auto login group by command line. In our case it was also necessary to edit a GDM config file to enable auto login.

In order to configure the Pi’s to shut down at a scheduled time we initially used the system control panel. However, this did not provide flexible shutdown options, such as setting a shutdown time for specific days of the week. Instead we chose to write a bash script to run at system start which determined the day of the week and ran a command to shut down at the desired time. This effectively enabled the Pi’s to shutdown and startup autonomously on a configurable schedule. We also set up a small data server which could be queried for the shutdown and startup schedule, however this was not in use when the project entered long term maintenance.

Remote Access and Domain Configuration

To limit remote access to authorized users we first joined the Pi’s to our local domain using realm. For this to work in our environment it was necessary to edit the configuration files for KDC as well as SSSD, and Pam to enable DNS resolution of hosts, LDAP authentication, and AD service discovery. This enabled us to join the Pi’s to our Active Directory controlled Domain and authenticate users with their network login credentials. The realm permit –groups command did not work for AD group authentication control due to a bug, making it necessary to manually edit the related configuration files. We were then successfully able to limit both remote and local logon rights to appropriate Active Directory groups. Additional system configuration limited sudo access only to System Administrators and Production Engineers.

We then configured the system to only allow SSH connections, and installed TigerVNC. In order to maintain secure access, the VNC server was setup as accessible from localhost only which could be accessed by using an SSH tunnel connection first which required Active Directory authentication. We configured the Pi’s to launch TigerVNC’s x0vncserver application to display 0 to give remote access to the live display when necessary. We also placed the Pi’s on an isolated subnet with limited access too and from the rest of the network, effectively locking down where remote sessions could be attempted from in addition to requiring authenticated login credentials.

Task Automation

Each Pi needed to be able to display information from multiple sources, while utilizing the full screen. Several of these sources required user authentication before they could be displayed. To meet these requirements we installed Mozilla Firefox and a password management utility. In some cases it was possible to setup authentication by private key file, however due to security concerns regarding the possibility of the key file being stolen from a computer not being used by a person while logged in this was not implemented at the time. Also, because not all display tasks that required authentication had this option, it was still necessary to use a password manager for automated login attempts.

The multiple tasks also required that only one task be displayed at a time. To accomplish this, and to maximize screen use we installed plugins for Firefox that forced the browser into fullscreen mode and a second plugin that automatically switched between tabs on a configurable timer. The browser was launched by script on a delay after system startup and auto login completed. In order for all of the correct web pages to be opened automatically on browser launch we took advantage of Firefox’s ability to have multiple home pages. By listing all of the web pages to be displayed in a pipe symbol delimited list, Firefox will open all listed web pages in separate tabs when launched.

Long Term Maintenance

In order to make the system easily maintainable long term on an enterprise scale several measures are needed. First, a basic Raspberry Pi drive image that has all shared tasks and applications installed and basic configuration should be maintained and kept up to date. Second, a configuration repository that stores settings specific to each Display’s tasks can improve maintainability. DFS did not fully implement this second maintenance measure before entering the long term maintenance phase of the project, however it was a goal of the IT department to fully automate these configuration tasks. Scripts were actively being written to automate configuration tasks after drive imaging for all desktop computers, in addition to installing and configuring a Microsoft System Center Configuration Manager[6]. There were also plans to implement similar measures for the Raspberry Pi’s after the System Center service had been fully deployed.

A minimum configured drive image was maintained, and backup copies were stored as well. The majority of other maintenance tasks were minor such as the occasional need to update software, change stored passwords or credentials, or configuring new display tasks.

Project Conclusions:

This project was completed toward the end of Spring in 2016 and went into long term maintenance. The Raspberry Pi’s continued to meet expectations, and required very little maintenance. Software developers were able to easily monitor development progress, unit and project compilation progress and results. The Help Desk/System and Network Administration staff used their’s to display Network Traffic stats and monitor incoming support tickets. The call floor Pi’s were not configured by the IT staff at DFS, but were provided by the company that also supplied their desktop IP phone software. As a result, the call floor displays could not be configured for multiple display tasks and only displayed call data.

The Raspberry Pi based displays provided by the IP phone service company provide a good baseline for success comparison. The call floor Pi’s did meet the basic needs of the call floor, displaying important aggregate call data in real time. However, because they were not configurable for multiple tasks, they were only able to supply a somewhat limited view of that data. With multiple sections of the call floor handling different tasks, configurable displays could have performed both tasks. There were 4 Pi’s displaying on the call floor, with minor configuration that was possible for our in house built implementation it would have been possible to guarantee that at least one display also showed the basic data, while periodically showing information specific to tasks performed in each section.

The Developer and Help Desk Pi’s were used to display additional tasks, including the results of the company Foosball tournament. The call floor also had its own events which could have been displayed by our configurable Pi’s. Instead, the call floor used white boards for displaying other information, much of which had to be displayed in multiple locations, requiring a person to walk to each board and change them every time a score or top personal sale’s number changed. For the company Foosball tournament a simple database was used to store team scores, and a Fandom Wikia page automatically displayed the tournament results on all of our Pi’s.

Our own configurable display solution was therefore highly successful in several respects. The cost to purchase a Raspberry Pi and all necessary parts was minimal, under $100 per Pi, resulting in saving thousands of dollars compared to just a few displays using full desktop computers and tens of thousands compared to purchasing displays with “smart” applications. Because they were highly configurable they provided the same flexibility of using a full desktop computer to control displays, allowing each display to meet multiple needs where “smart” apps built into displays and display modules provided by third party companies could only display a single task per screen. For these reasons we considered this project to be a success.

Technologies We Used to Complete this Project


References

[1] http://www.sap.com/product/enterprise-management/business-one.html
[2] https://www.raspberrypi.org/
[3] https://fedoraproject.org/wiki/Raspberry_Pi
[4] https://www.nagios.com/products/nagios-network-analyzer/
[5] http://www.directfinancialsolutions.com
[6] https://www.microsoft.com/en-us/cloud-platform/system-center-configuration-manager
[7] https://en.wikipedia.org/wiki/Delimiter