Project Honolulu – First impressions

Project Honolulu is Microsoft’s attempt at revamping the server administration experience.  Historically the Windows server toolkit has been built around using numerous MMC (Microsoft Management Console) plugins – things like Event Viewer, AD Users and Computers and DNS Management are all built on MMC.  We’ve seen a couple of attempts at revamping this in the past, there was Server Manager in 2008 and a refreshed form in 2012.

I suspect one of the driving forces behind Honolulu is the shift from RPC-based connectivity to WinRM for remote administration of servers.  Honolulu seems to represent an alignment with this since it supports only Server 2012 onwards as nodes to manage, and its gateway component installs on Windows 10 or 2016.  The documentation claims the management functions are performed using remote powershell or WMI over WinRM.

 

Installation

The installer for Project Honolulu is only about 30MB.  While it supports installation on a system in a workgroup configuration, the TrustedHosts value for WS-Man needs to include the target nodes to be managed.  The installer can do this for you or you can manually perform the commands to do it.  On a domain joined system this isn’t required.

 

Starting Up Project Honolulu

Following install, the web interface will load with a tour splash screen.  The main landing page is bare to start with.

Clicking on the Add button presents three possible options – a regular server connection, a fail-over cluster or a hyper-converged cluster.  The first 2 options then allow the adding of single items or importing in bulk from a test file.  The hyper-converged cluster option allows only single items.  When adding in the server name or IP, Honolulu will appear to perform ongoing checking of the name to ensure it meets correct requirements and if it can connect with current credentials.

Single sign-on authentication is supported and is the default option.  In the case of my test scenario, the system running Honolulu was in a workgroup and attempting to connect to domain based systems.  For this, I manually specified credentials.  One thing to note is it appears that even if a server is added using IP, the name will be resolved (perhaps by the initial connection) and subsequent connection attempts will fail if this name can’t be resolved by DNS.  For domain-joined installations this should be  a very rare case, in workgroup configurations it could happen.  Honolulu will initiate its connections to the target node on the target’s port 5985.  Assuming initial connection and authentication is successful, you should see a HTTP/1.1 Status 200 in a network capture.  If everything is good, the status will have a tick and say “Online”.  From there you can click on the server’s name, or select it and click Connect to drill down further.

 

Overview Page

The first page shown is an overview of the server and includes some metric graphs like CPU and RAM usage and the ability to shutdown or reboot the server remotely.

On the left is a collapsible menu of the other sections such as events, firewall, registry, processes, etc.

 

Tools

Some of the tools seem quite functional.  The Events one allows filtering of the event log with a reasonable number of criteria and allows exporting.  Files allows browsing of the target node’s file system with the ability to create folders, rename/delete and so on.  The Process section looks to have all the main functions you would typically use in Task Manager, allowing remote termination of processes.

 

Closing Thoughts

Project Honolulu is an interesting tool from Microsoft.  It seems capable of replacing the traditional Server Manager app that most Windows sytem administrators are familar with.  I’ll be most interested in seeing how it develops in terms of extensions.

VCSA 6.5 Root Account Password – Reset and Cause Investigation

One of the more frustrating experiences one can experience with VMware’s vCenter Server Appliance (VCSA) is having the root account locked out or forgetting the password for it.  I recently experienced this after I rebuilt the VCSA in my home lab from scratch.

How to Reset the VCSA Root Password

VMware have a short process on how to reset the password for the root account, detailed in KB2147144.  THe process is:

  1. Backup the VCSA (via snapshot or other means)
  2. Reboot the VCSA
  3. During the boot process, when the photon splash screen appears press the e key to get into the boot menu
  4. In the text box that appears, go to the line starting with “linux”.  Go to the end of the line (which is right after the text “consoleblank=0”) and enter the text rw init=/bin/bash  This will cause the boot process to jump right into the bash shell without needing credentials
  5. Press F10 to continue booting
  6. At the command prompt, run the passwd command to reset the password
  7. Unmount the file system by running umount /
  8. Reboot the VCSA using reboot -f
  9. Following reboot, confirm the new password works
  10. If you took a snapshot in step 1 remove it

 

Cause Investigation

After resetting the password and restarting, I still couldn’t login.  One thing I noticed was there had already been 20 login failures.  In my situation, the VCSA was working one evening and the following morning the login issues happened.  Something had to be causing these issues.  Using the password reset process to get to the bash shell again, I looked around in some logs.  First I tried checking the /var/log/messages log.

Failed Logins search using grep
Failed Logins search using grep

Unfortunately, using ‘FAILED’ only showed 2 login attempts on the console which were caused by myself after the lockout happened.  Failed SSH login attempts are logged under a text string that uses ‘Failed”.  The second search attempt used ‘Failed’ and yielded better results.

Search results using 'Failed'
Search results using ‘Failed’

The log had numerous entries for 192.168.1.55 trying to login using root and other accounts.  The system on 192.168.1.55 was the trial of Nexpose.  Even though I didn’t have credentials set in Nexpose to logon to the VCSA, it still was trying to logon using root and was causing the failures.  This hadn’t been an issue prior to the VCSA rebuild.

I excluded the VCSA from the scanning that Nexpose performs and did the password reset process again.  I was now able to login successfully.

 

Adding extra service mappings to Nexpose

Nexpose does have good coverage of services in the “well known” range of ports (0-1024).  An environment with a lot of propriety systems will cause Nexpose to some services as unknown or even misidentifying them.  The screenshot below is a good example of this.

Nexpose Services Data
Nexpose Services Data

The example is from a Domain Controller.  Nexpose identifies the port 3389 service correctly as RDP.  Ports 3269/32769 are used by the Global Catalog service, so labelling them as LDAP/LDAPS isn’t strictly accurate.  For port 3260 and 5666 it gives up.  Depending on your needs, you may want to get these labels a bit more accurate.  This can be achieved by using a custom service names file (you can alter the default one, but it’s probably best to leave that in its default state).

The default file, default-services.properties, is located in the <install location>/plugins/java/1/NetworkScanners/1 folder.  The format is basic, with each line as <port #>/<tcp or udp>=<Service Name>.  Some of the custom ports I added are shown below:

Custom Service Mappings
Custom Service Mappings

 

Once the properties file is in a state you’re happy with, place it in the same folder as the default one and either create a new or edit an existing scan template and put the file name into the field on the Service Discovery section.  Load up the page of an asset to test and queue a scan on it with the scan template.  The reported services should update with the new values.

If all your IT department does is BAU work, you have a problem

Over the years I’ve heard stories from peers and read them online about the very ordinary experiences they had as IT workers.  I’ve even experienced it myself.  A common theme is that the IT department is only performing a maintenance function, often called Business as Usual (BAU), operations or break-fix work.  Being in that position can be very dangerous.

Read more