SCGOV Internship

Report

By: Colin Harvey

Introduction

I would first like to point out that I believe I speak for both of us when I say we greatly

appreciated the opportunity of working with Sarasota County. For the past eight weeks Collin

and myself have been shadowing the daily work flow of Sarasota County's EIT department.

Neither of us knew what to expect from this opportunity because we had little knowledge on

how local government worked. We learned very quickly that anything involving government

becomes very convoluted. There are so many facets that go along with the EIT level and even

more so when involving government. Within this report I would like to explain these facets of IT

and how our local government utilizes them.

Network

The first thing on my list is the network as it is really the backbone for everything if you

think about it. Sarasota County isn't just a single entity as some may think; the county store,

route, switch and protect data for: Sarasota County Government, School Board, Memorial

Hospital, County Clerk, Tax Collector, Sheriff's Office, Property Appraiser, State Attorney, Public

Defender, Town of Longboat Key, Emergency Services, and 911. It is imperative that the

network is not obstructed as more than just one entity will suffer so the county must ensure

speed, capability and security of the network. The county utilizes mainly Cisco switches and

routers for data transmission and have three internet service providers to ensure connection:

FPL, Comcast and Verizon. The network connections are comprised of copper, fiber optic and

some wireless. The Suncoast Regional Data Center is where all of these connections lead to or

from and houses all the data running on the county network.

1

During our week spent with the Network team we were given a brush up course on the

basics of networking as they described their network setup. In addition to these "mini-lessons"

we were also given a lab of sorts to work on setting up a network of our own. We were given

two switches, two routers, and a laptop in which we created two different networks that could

ping to each other.

Security

Security from external threats is a must in today's society and especially so for any part

of the government that has information to steal or even utilities to hack. At the Sarasota County

administration building we were introduced to the four gentlemen in charge of EIT security.

Each day we cycled through the security team to get a feel for how they keep EIT secure from

the inside and out.

Sarasota County uses a multitude of different applications and software to keep their

network secure. We were first introduced to Blue Coat proxy which provides control of web

traffic. the Blue Coat server acts as an intermediary between the computers in the network and

the outside world, allowing or disallowing access to certain websites. To protect the other

employees in the building from themselves, the security team blocks all malicious sites to

prevent accidental infections to the network. This includes most social media or forum sites like

Facebook which is infested with misleading links that are often a wolf in sheep's clothing.

In addition to Blue Coat the security team runs Check Point firewalls as an affective

intrusion prevention system that blocks all unwanted incoming traffic. There are two firewalls

for EIT, school board, 911 call center, and the virtual private networks that hold some 80

2

remote sites. Redundancy is a key factor in the IT business to prevent security downtime if

something were to happen to one of the firewalls which is why there are two for each.

It is said that the best offense is a good defense and even better is to be proactive in

your defense. The security team uses Nessus Vulnerability Scanner to look for exploits within

the network such as open ports that should be closed. In addition to scanning the local area it

can also be deployed to remote sites as well. Nessus is quite aggressive and most be tuned

down after installation as it will block certain ports and applications by default. The data

retrieved will be transcribed to an excel spreadsheet via the "pars nessus xml perl script" that

they use as the Nessus data transfer application is a fortune. The security team also uses MSUS

or Microsoft Server Update Scanner to pull Microsoft server updates into a log so the team can

download and integrate into the system on their own time which is usually on the weekends to

keep low traffic during work hours. Another benefit of being able to choose which updates to

install is to avoid possible vulnerabilities that security updates carry. System administrators will

take the updates and apply them to one workstation to test before applying network-wide.

Another component to the security team's arsenal is the F5 switch. This is the

application layer firewall meaning it only acts on layer 7 of the OSI model. It is also a load

bouncer which helps to direct traffic from overloaded or slow server to others than can handle

the influx.

It is said that the best offense is a good defense and even better is to be proactive in

your defense. The security team uses Nessus Vulnerability Scanner to look for exploits within

the network such as open ports that should be closed. In addition to scanning the local area it

3

can also be deployed to remote sites as well. Nessus is quite aggressive and must be tuned

down after installation as it will block certain ports and applications by default. The data

retrieved will be transcribed to an excel spreadsheet via the "pars nessus xml perl script" that

they use as the Nessus data transfer application is a fortune. The security team also uses MSUS

or Microsoft Server Update Scanner to pull Microsoft server updates into a log so the team can

download and integrate into the system on their own time which is usually on the weekends to

keep low traffic during work hours. Another benefit of being able to choose which updates to

install is to avoid possible zero-day vulnerabilities that security updates carry. System

administrators will take the updates and apply them to one workstation to test before applying

network-wide.

Email is a big deal at the county as it is the primary form of communication between

employees. Unwanted email can be one of the biggest threats to an organization since you

can't control what your employees will click on but only advise them. This is why the security

team uses a program called Solar Winds as their external email or postfix which intercepts all

the incoming mail and decides what emails get through. The only constraint on the acceptance

of mail is the size of attachments which can be let through; anything 200Kb or less is rejected.

This is due to the assumption that most attachments with this little of memory contain

malicious scripts within them. Once mail has been accepted through postfix it then enters the

internal host and makes its way through Spam Assassin. A score is given to the mail based off of

the security team's constraints and if a piece of mail gets a score greater than five then spam

assassin labels the message as such. If something does happen to make it through all these

4

preventative measures then McAfee antivirus is available on every workstation in the county.

They also use ClamAV for the few Linux servers they operate.

With all of these security applications protecting the network it can become a

cumbersome task to figure out how to trouble shoot an issue when it arises. In order to

streamline work efficiency the EIT department at Sarasota County uses an enterprise tool called

Splunk for logging of system data. Any information that would be pertinent to an issue on the

network, active directory, servers, databases, and applications like blue coat can be found in

Splunk logs granted they are passing through Splunk. Examples relevant to the security side:

Packet flow data which yields performance degradation, timeouts, bottlenecks or even

suspicious activity on the network

Database audit logs that reveal how data on a database was modified over time and

who made the changes

Windows events such as updates for business critical applications which can sometimes

lead to vulnerabilities

Active directory changes which could possibly be an attacker elevating privileges

In addition to the logging of events on a system, user documentation is also very crucial. We

learned that even if working alone it can become a tedious task to have to look back through

your work from the day prior to try and remember where you left off or what you changed last.

The security team also utilizes some procedures and policies in order to create a more

secure environment. Not every employee at the county administration building is considered

'tech-savvy' which is why a password policy is in place to keep people from utilizing guessable

5

passwords like '123abc'. The policy sets the constraints for a password to be eight or more

characters long, must include a number and must include a symbol. Policy management is also

another big part of proactive defense as policies can become inactive or stop working due to

security updates from Microsoft. Checking for password changes out of the scope range or

passwords not meeting the constraints could mean that the policy is not actively doing its job.

In some cases the policies could even be exploited by employees.

Example:

Scott from security and Sue from system administration ran into an issue where an

employee in the building was calling System Admin (Sue) and complaining that she

couldn't log into her user. After Sue exhausted her options she asked Scott to take a look

at the employee's password to see what it was as Sue assumed the employee tried

changing it. They soon found out that the employee was lying about trying to change

her password as there were 11 password change attempts on her user account. This

employee figured out that the password policy was set from 0 - 90 days and that the

password counter was 10. She could then circumvent the password counter of 10 by

changing her password 10 times as quickly as she wanted and use her same password

again. This employee did all of this work just to use the same password again and shows

how a policy in place may not be secure enough.

Scott will also do password auditing on the list of cached passwords from all employees to find

the ones that are following the policy constraints but aren't doing their part and making the

password constraint live up to its potential. Scott will run John the Ripper for brute force and

6

dictionary attacks to find the weakest passwords. He then uses Metasploit hash tables to crack

the harder passwords which still need work. The hash table is also a good way to check for

previously used passwords as a hash table is comprised of numerous previously used password

hashes and compares them all against one password hash to find a match.

Keeping the network secure is imperative for organizations these days and while EIT

security systems like Blue Coat do an amazing job, there are still rudimentary level precautions

that need to be taken. At the Sarasota County administration building the majority of the doors

are accessed only by badges (show badge). These badges allow access to different levels of

security depending on your clearance. The blue coated badges are your basic level access cards

that get you into public work areas and amenities like the snack room or gym. Some blue

badges can have higher privileged access depending on what you need to satisfy your position.

Red coated keycards are security clearance and can access areas with sensitive information like

the data center.

Sarasota County is working PCI Compliance level 3 of 4 in order to guarantee credit card

information security.

Investigations happen from time to time when an incident occurs involving a

workstation. In these cases security will work with human resources by gather information from

an image they took of the workstation then providing human resources with a report on what

they found so an action can be taken against the employee.

Example:

7

Scott said that there had been multiple cases of employees taking home their work

laptops and using them for extracurricular activities. There was even a case where an

employee took her laptop home over the weekend and either her son or husband used

it to visit inappropriate websites which of course were still in her internet cache and

history when she returned to work the next Monday. After reviewing this case security

and HR came to the conclusion that the woman had no idea and they took no action

against her. As much of a joke as it may seem, the county takes security very seriously

and I was even told that you could get away with coming to work drunk or even high

and you would be sent home or to rehab whereas if you were caught breaching security

by simply visiting malicious sites you could be fired on the spot which has happened.

The security at Sarasota County isn't perfect as this is quite difficult to ascertain given all

the avenues for attack today but it is definitely a force to be reckoned with. When Clifton

Larson Allen auditing firm came to try and penetrate their defenses, it took eight attempts to

succeed and it wasn't even their security that let them down in the end, it was an Elected

Official with elevated privileges who let the penetration testers in by downloading an '.exe' file

from an email that was sent by 'Scott Gibbs' from IT Security. In their defense it was quite tricky

as they spoofed Scott Gibbs' email and changed the hyphen to an underscore which at first

glance for many would be difficult to catch. Overall, the auditing firm stated that Sarasota

County "implements appropriate configuration standards & policies" and that other systems

should aspire to live up to those same security standards.

8

The security at Sarasota County isn't perfect as this is quite difficult to ascertain given all

the avenues for attack out today but it is definitely a force to be reckoned with. When Clifton

Larson Allen auditing firm came to try and penetrate their defenses, it took eight attempts to

succeed and it wasn't even their security that let them down in the end, it was an Elected

Official with elevated privileges who let the penetration testers in by downloading an '.exe' file

from an email that was sent by 'Scott Gibbs' (IT Security). In their defense it was quite tricky as

they spoofed Scott Gibbs email and changed the hyphen to an underscore which at first glance

for many would be difficult to catch.

System Admin

The system administration team was the biggest of the teams within the Sarasota

County EIT department and rightfully so. They mainly dealt with maintaining the County's email

servers, active directory environment and anything to do with VMware. In addition to

maintaining the servers via the network, they also help to deploy the physical servers which we

were able to help them with when they transported 911 call servers from the Administration

building to the EOC.

As I stated in the security section Email is a significant part of the daily grind within the

County. The county uses Microsoft for everything so as a result they use Exchange Server 2007

for their email system. As of right now they are getting ready to upgrade to Exchange Server

2013 but are dealing with the legalities of email archiving; they are required to have every

email saved all the way back to 2007 but are running out of space. Exchange has a rule set that

9

will drop any mail labeled as spam from Spam Assassin. They also have a transport rule set that

sends any emails between commissioners to public records which are kept for five years.

The County utilizes VMware virtual machines for their servers. They utilize vCenter by

VMware as a way to manage their 16 blades that hold 89 virtual servers within them. These

virtual servers are comprised of SharePoint, web servers, security, time keeping, permitting,

and basically everything else except for the Exchange server are virtual. The benefits to

virtualization include cutting down on space, heat, time, and money of course.

Active Directory is the County's way of protecting the employees from themselves by

providing an effective means of access control. Active Directory is a collection of computers and

servers within the Microsoft environment utilized by Sarasota County. It's job is to associate

users, groups, workstations, applications, servers, and provide security by putting users within

groups and then applying group policies to the groups. By adding users to groups you are

eliminating the individual administering process. Examples of policies could be things like,

10

locked icons on desktop, no internet access, or privileges.

The IT security team at the Sarasota County building works within the BCC network. From what

Scott said BCC is in the same forest (SCGOV) as Clerk of Court, Sarasota County Sherriff's Office,

and Sarasota County Property Appraiser but they just advise them security wise. The security

team has no reigns or control over the other three networks. Judicial, Elected Officials and Tax

Collector are all in separate forests and are accessible by the inhabitants of SCGOV. The SCADA,

911, and ADFS Service forests are blocked by firewalls to prevent access. Temporary access can

be gained if connected to the ADFS service which will allow access to the firewall blocked

forests via a nonce token.

Applications and Data

Out east of the interstate on Fruitville there is a building referred to as B.o.B or "Big ol'

Building" which is a warehouse converted into a sea of cubicles. There are many groups and

services within B.o.B but we focused on three team applications: GIS, Maximo and Amanda.

11

During our stay at the B.o.B we were introduced to the managers of each of these three groups

and learned how each of these applications have a hand in streamlining processes within the

county.

GIS stands for geographic information system and is an application that is used to

capture, store, manipulate, analyze, manage and present different layers of geographical data.

To get an understanding of what this is you can think of the GIS application as an in depth

version of Google Earth which works much in the same way but with different layers. When

looking at Sarasota County within GIS, the base level is just as you would expect to see on

Google Earth which is just a visual representation of Sarasota as an above view shot. This above

view shot is referred to as an ortho-image and is quite detailed for the distance at which it is

taken. Layers can then be applied to the map view of Sarasota county to help the county make

decisions on certain matters like being prepared for a flood. Examples of some layers on GIS are

things like flood zones, emergency routes, school zones, conservation areas and demographics.

I only listed the main layers they showed us but they actively maintain around 300 different

layers and another 100 layers are static. Certain layers, like demographics for example, thrive

on census data to create these visual representations.

In addition to the ortho-imagery on the GIS application, there is also an online

application called Pictometry that Sarasota County uses that is available for those that need to

utilize oblique-imagery which produces images that give side views to buildings and structures.

The application allows you to measure building dimensions by outlining the building with a tool

which is especially important for property appraisers so they can take measurements or

12

evaluate the status of a building right from their desk. Sarasota County just signed a six year

contract for Pictometry to do fly-over pictures at a resolution of four inches to every pixel. GIS is

also used to log the coordinates of assets which are tracked and managed by the application

Maximo.

Maximo is an application created by IBM for asset tracking and management. Basically

Maximo holds all the asset data for a specific parsec or address and gives information on what

it is, when it was put in, how old it is, and its status. Maximo also holds all of the purchased

supplies and utility parts to be used in the field as they are assets. With this in mind, before

every project the Maximo team must use the application to figure out what assets they already

have, what they need to accomplish the project, what they have to accomplish it, and the cost.

All in all, Maximo is used as an:

inventory module warehouse control

purchase module for inventory

tracking module for inventory, products and assets

asset module to create asset records (model / manufacture)

In addition Maximo and GIS work together; GIS holds the geographic position of the assets and

Maximo labels and manages the assets. This is done by taking a GPS pole equipped with a

YUMA device, which is a ruggedized tablet with windows 7 OS, and placing it over top of the

asset while creating the asset label and location in Maximo.

Examples of assets:

13

Some of the main asset areas are parks, traffic, water, transportation, irrigation, sewage,

and storm water

Storm water assets include things like manholes, drainage ditches, retention ponds,

gutters, etc. ; water pumps and lift stations are 'water assets'

Trees were one of the first types of asset to be listed in the GIS and Maximo

applications. The trees are considered assets and if one is to be knocked down in a

storm, FEMA will pay for a replacement based on the age, size, and species.

Maximo is also very good with keeping data integrity by not allowing new data to be written

without going through an editing process and also doesn't allow data to be changed without

approval. Individual edits come first then quality assurance and finally becomes default.

Amanda is a business process automation application for government. Everything in

Amanda deals with processes up for review which handles people, properties and processes. In

other words Amanda streamlines the permit process. Amanda's footprint is enterprise wide and

includes things like security & bonds, escrow accounts, utility permitting, construction

permitting, zoning, code enforcement, contractor licensing, property records and more. There

is also a front end portal that allows users to look up permits as they are permanent which is

good for inspectors or supervisors of work sites.

From a security standpoint, each of these applications (Amanda, GIS, Maximo) uses its

own form of Active Directory so that they aren't at the mercy of the SRQ Security team. All

joking aside, by adding the B.o.B applications to the security team's long running list would be a

14

tremendous amount of work considering the amount of lockouts that could happen between all

three of the applications due to updates.

Netmotion is an application that is quite useful for entering data to Maximo and other

applications. Netmotion allows data to be entered into applications and saved on the user end

while seamlessly transitioning to new wireless connections and staying connected to the BCC

domain. This is invaluable for county workers that will use devices like the YUMA tablet and

GPS pole to record data while in the field and not have to worry about whether or not the data

was sent to the server.

The HELP! application is a basic ticket queue system that was built in house. Pretty much

every area in EIT uses it: Security, System Admin, Telecom, Apps, Vitil, Networking and many

others use this application as a way to streamline troubleshooting of issues revolving around IT.

The county utilized many other small applications such as Recware Client and PC Res.

Recware is used by parks and recreation as a way to sign up for summer camps reserve park

utilities while PC Res acts as an interface between the user and the desktop of a library

computer that requires you to scan your library card to reserve computer time. PC Res is only

one of about seven applications Fruitville Library uses and there are around 240 total

applications used by the county so this is just the tip of the iceberg.

Vitil Solutions

Vitil is the company that Sarasota County outsources for workstation maintenance. The

field techs take care of: re-imaging workstations, break-fixes, updating and installing

15

applications, and supplying loaner workstations. They are a Dell distributor which is good

because the county purchases mostly Dell products. The field technicians service the

workstations at Fruitville Library, Emergency Operations Center, B.o.B, Little B.o.B, and the

County Administration building.

The field technicians utilize the System Center Configuration Manager to store

programs, applications, updates, and drivers that can be downloaded onto workstations by

plugging in the appropriate 'boot-from' flash drive and connecting to the server to download.

The SCCM is a great way to streamline template configurations and installations for

workstations instead of doing it one at a time which takes hours. Just like the rest of the EIT Vitil

makes use of the HELP! system to take requests of service. During our day with a Vitil field

technician we were given the opportunity to re-image a workstation which utilized the SCCM

process I explained. We did a refresh on a workstation which is basically switching out old

components such as mice, keyboard, towers and monitors. We swapped the mouse, keyboard,

and tower then moved the user accounts to the new computer. All of these Dell components

are kept in a warehouse directly next to the Vitil Solutions help desk office. This warehouse has

just about any tool or part that you can think of to repair, re-image, or update workstations.

The help desk is made up of six employees and is the frontline for Sarasota County

taking first level calls. When an issue is relayed to one of the help desk employees they first go

to their knowledge base application called "Knowledge Collaboration" to see if this issue has

occurred before and if so how to respond. If a problem is easily fixed by a help desk employee

they can either guide the user over the phone or remote into workstations of Sarasota County

16

employees. If a problem is too specific or in depth for the call center they will route the issue to

the appropriate team as a ticket via the HELP! application.

GovMax is another of Sarasota County's in house applications which handles fiscal

budgeting for the county and municipalities. Built from the ground up at Sarasota County,

GovMax is now on version 6.0 and is being utilized by 19 different counties across the United

States. Sarasota County couples GovMax with Business Objects to report data to users over

GUI. Crystal Reports is then used to make reports of the data that is collected within Business

Objects which include things like land development and government spending.

Project Management / Financial

Project management is a significant part of the work flow at Sarasota County as there

are numerous projects to be taken on each week. Business analysis is a core component of this

and is described as the liaison between the tech and business world. To be more specific they

keep track of the scheduled events, budget, and scope of projects while keeping in mind

possible scope creep and how to mitigate it. From what we were told there are only about four

people who deal with business analysis within the county administration building and they are

also project managers. The county relies heavily on project management to complete large

projects that affect citizens like fixing roads, land development, and construction. Project

management is heavily utilized within IT, construction, and new product development.

17

Before a project is started within IT they have to account for the amount they are

spending per year. Cost models are created that weigh the differences per full time employee,

workstations, enterprise wide programs, services, and maintenance. They are an internal

service so they don't take money from general funding to pay for IT related needs. To keep

workstations in top condition they must also plan out a refresh cycle to allocate spending in

small amounts as opposed to a huge lump sum. The refresh cycle generally lasts about four

years and each week they refresh about sixteen computers. TO offset their budget, the county

EIT department rents out space in the data center and also has GovMax which is used by

municipalities.

Conclusion

Our time at Sarasota County was an amazing experience and it was an honor just to

participate in the internship. It was fascinating to see the County’s EIT department handle the

myriad of IT needs. I think we both learned a great deal about what it is to have a career in IT

and do it well. It was also very eye opening to see how involved IT is even in a smaller county

like Sarasota. It was a wonderful experience and I would highly recommend it to any other USF

student.

18