VCSA 6.5 Root Account Password – Reset and Cause Investigation

17/08/2017 by user

One of the more frustrating experiences one can experience with VMware’s vCenter Server Appliance (VCSA) is having the root account locked out or forgetting the password for it. I recently experienced this after I rebuilt the VCSA in my home lab from scratch.

How to Reset the VCSA Root Password

VMware have a short process on how to reset the password for the root account, detailed in KB2147144. THe process is:

Backup the VCSA (via snapshot or other means)
Reboot the VCSA
During the boot process, when the photon splash screen appears press the e key to get into the boot menu
In the text box that appears, go to the line starting with “linux”. Go to the end of the line (which is right after the text “consoleblank=0”) and enter the text rw init=/bin/bash This will cause the boot process to jump right into the bash shell without needing credentials
Press F10 to continue booting
At the command prompt, run the passwd command to reset the password
Unmount the file system by running umount /
Reboot the VCSA using reboot -f
Following reboot, confirm the new password works
If you took a snapshot in step 1 remove it

Cause Investigation

After resetting the password and restarting, I still couldn’t login. One thing I noticed was there had already been 20 login failures. In my situation, the VCSA was working one evening and the following morning the login issues happened. Something had to be causing these issues. Using the password reset process to get to the bash shell again, I looked around in some logs. First I tried checking the /var/log/messages log.

Unfortunately, using ‘FAILED’ only showed 2 login attempts on the console which were caused by myself after the lockout happened. Failed SSH login attempts are logged under a text string that uses ‘Failed”. The second search attempt used ‘Failed’ and yielded better results.

Search results using 'Failed' — Search results using ‘Failed’

The log had numerous entries for 192.168.1.55 trying to login using root and other accounts. The system on 192.168.1.55 was the trial of Nexpose. Even though I didn’t have credentials set in Nexpose to logon to the VCSA, it still was trying to logon using root and was causing the failures. This hadn’t been an issue prior to the VCSA rebuild.

I excluded the VCSA from the scanning that Nexpose performs and did the password reset process again. I was now able to login successfully.

Adding extra service mappings to Nexpose

10/07/2017 by user

Nexpose does have good coverage of services in the “well known” range of ports (0-1024). An environment with a lot of propriety systems will cause Nexpose to some services as unknown or even misidentifying them. The screenshot below is a good example of this.

The example is from a Domain Controller. Nexpose identifies the port 3389 service correctly as RDP. Ports 3269/32769 are used by the Global Catalog service, so labelling them as LDAP/LDAPS isn’t strictly accurate. For port 3260 and 5666 it gives up. Depending on your needs, you may want to get these labels a bit more accurate. This can be achieved by using a custom service names file (you can alter the default one, but it’s probably best to leave that in its default state).

The default file, default-services.properties, is located in the <install location>/plugins/java/1/NetworkScanners/1 folder. The format is basic, with each line as <port #>/<tcp or udp>=<Service Name>. Some of the custom ports I added are shown below:

Once the properties file is in a state you’re happy with, place it in the same folder as the default one and either create a new or edit an existing scan template and put the file name into the field on the Service Discovery section. Load up the page of an asset to test and queue a scan on it with the scan template. The reported services should update with the new values.

If all your IT department does is BAU work, you have a problem

16/01/2017 by user

Over the years I’ve heard stories from peers and read them online about the very ordinary experiences they had as IT workers. I’ve even experienced it myself. A common theme is that the IT department is only performing a maintenance function, often called Business as Usual (BAU), operations or break-fix work. Being in that position can be very dangerous.

VMware NSX – First Impressions

29/06/2016 by user

One of the first “killer applications” on the PC platform was Lotus 1-2-3, a spread sheeting program that greatly improved the productivity of the people using it and making a clear case for buying PCs. More recently, we’ve seen this sort of thing happening in IT infrastructure, with virtualisation, automation, cloud and “as a service”. VMware’s NSX product is the latest in a line of products from VMware in this sort of area.

If we go back to the “good old days” of getting a server up and running, it could take weeks. The diagram below shows the amount of effort involved.

While some of these numbers may have been more or less depending on circumstances, in many cases it could’ve taken over 150 business hours to get a server ready for use. Or almost a full month.

VMware Horizon 7 – First Impressions

06/04/2016 by user

VMware Horizon is a Virtual Desktop Infrastructure (VDI) product which initially allowed provisioning of Virtual Desktops off a base image in an easily to manage fashion. Over time, VMware have added extra functionally, such as the ability to add Remote Desktop Services (RDS) servers.

Version 7 has added a number of interesting features and improvements. The one mentioned first in the release notes is Instant Clones. This is a technology I’ve been following for a while, ever since I read about it. Originally known as VMFork, it’s a technology to allow very rapid, almost instant, provisioning of Virtual Machines. Duncan Epping wrote a good overview of VMFork/Instant Clone back in 2014. Support for Virtual Volumes and Linux desktops are some of the other features that have been added.

Results from Telstra’s Free Data Day

04/04/2016 by user

Telstra had another free data day on Sunday 3rd April and this time I decided to take advantage of it since my phone has been capable of giving back speed test results of 90Mbits/sec compared to 12 on my ADSL.

Prep & Setup

Unfortunately I made the assumption that my wifi adapter would work again since the Windows 10 upgrade I performed on my PC. I went to bed early and set the alarm so I would wake up around 3am. Due to issues with getting the adapter working, I lost about an hour and started around 4:30am. The primary aim was to download a number of games on Steam that I hadn’t downloaded yet.

SQL As A Service Proof of Concept with SQL 2012 and vRealize Automation

05/03/2016 by user

Standing up a redundant/highly available database infrastructure can be one of the more complicated pieces of work. Doing it by hand is a long process with any points where errors could happen. It was with this in mind that I decided to use this as my first “project” with vRealize Automation.

A Brief History of SQL Server High Availability

When discussing redundancy or high availability (HA) for databases, there’s two distinct outcomes – firstly to ensure the continued delivery of the service in the event of infrastructure failure (the actual HA part) and secondly to ensure the data is kept in an orderly fashion (data integrity, no loss of data, etc). Where these two activities happen depend on the technology used.

In older versions of SQL Server, these outcomes were achieved using SQL Clustering. In SQL Clustering, the HA function was achieved at the server level by having 2 or more servers, while data integrity was maintained by the database residing on shared storage.

Want hands on with Virtual Volumes? EMC has a Virtual Appliance for that

12/02/2016 by user

Virtual Volumes (VVols) is a new method of managing storage introduced in VMware vSphere 6.0. Unlike many of the new features in vSphere 6.0, VVols requires not just vSphere 6.0 to work, but a storage device that supports the technology. Fortunately, EMC have produced a virtual appliance that emulates a storage device with VVol support, so you can get some practical exposure to Virtual Volumes without needing a shiny new storage array. Download and documetation can be found at http://www.emc.com/products-solutions/trial-software-download/vvols.htm The process for getting Virtual Volumes completely working is rather long, as the flow chart from EMC’s documentation below shows:

I’ll run through the steps in getting the Appliance and Virtual Volumes working on a vSphere installation.

vRealize Automation 7.0 – First Impressions

29/01/2016 by user

vRealize Automation is, as VMware puts it, cloud automation software. It’s the black box where the magic that happens between a customer or consumer of your cloud services and the infrastructure the cloud sits on, providing the services we would normally associate with a cloud service such as self-service, elasticity and multi-tenant support.

vRealize Automation - The Black Box — vRealize Automation – The Black Box

In the past, this product was known as vCloud Automation Center, or vCAC. It was rebranded along with a number of other VMware products under the vRealize banner. However, the newly branded vRealize Automation product still retains some references to vCAC.

Travel Report – Cairns, January 2016

26/01/2016 by user

I had decided last year that I wanted to travel more during the course of the year, but repeating the sort of trip I usually do around November was out since it ends up being very expensive and time consuming. So the plan was to bounce over to the east coast of Australia or maybe to a close overseas location now and then for a break.

As it happened, QANTAS had a sale offer to Cairns that was valid over the Australia Day long weekend. So I went for it, booking as economy and putting in upgrades requests. 3/4 of the requests were granted automatically with the final leg (Sydney to Perth) sitting in “Confirmed” status.