United States

The Ultra-Secure Network Architecture


Meeting the nationally legislated security requirements of Sarbanes-Oxley (SOX), the Gramm-Leach-Bliley Act (GLBA) or business requirements of Visa and MasterCard is not optional. It is also complex and confusing, as many regulations overlap, and some are referred to generally under one law but more specifically enforced under another. Are you in compliance? If not, significant financial penalties can be incurred by your organization and, in the case of SOX non-compliance, it could even mean jail time. For eCommerce and other transactional environments that handle private information, ultra-secure network architecture will ensure you are in compliance.

The inadequate design of many Web-based applications configure a minimum number of servers in the demilitarized zone (DMZ) and use a Web server running Apache or Microsoft’s Internet Information Service (IIS), providing both HTTP (unencrypted) and HTTPS (encrypted) Web pages and Web applications on one server that can access sensitive information stored on non-DMZ systems.

Basic Internet Network Architecture Practices

The typical architectural diagram shown below offers only two slim layers of protection, yet it is widely accepted that more layers equal a more secure environment. In the diagram below, an attacker must compromise only one server to gain access to the Web applications provided on the same system.

Additionally, the basic Web-based network architecture does not protect against application attacks (e.g. buffer overflows or injections), since its primary source of protection are network firewalls. As application attacks gain in popularity, network-based firewalls will be insufficient to effectively prevent attacks from this new threat. While some network firewalls have application firewall capabilities, most security experts consider these underpowered offerings less protective than the single-purpose application firewalls that are available. In fact, network firewalls cannot protect custom Web applications at all.

Another issue basic implementations have is their default use of well-known tcp and udp ports for communicating. Unfortunately, many organizations’ Web applications are packaged solutions, leaving the organization unable to change the prescribed ports. As a result, once systems in the DMZ are compromised, the attacker can easily compromise other systems because of the default tcp/udp ports. Also, systems in the DMZ enjoy little to no monitoring or security controls.

Only one server must be compromised before accessing the Web applications is possible.

Because of these shortcomings, the basic architectural approach no longer provides the level of security now being demanded by the VISA Cardholder Information Security Program (CISP) and Payment Card Industry (PCI) security standards, Federal Information System Management Act (FISMA), SOX, GLBA and other regulatory and industry security standards and compliance efforts. The ultra-secure network architecture would.

The Ultra-Secure Network Architecture

The diagram below represents the base-level ultra-secure network architecture, which meets all regulatory requirements and limits the likelihood of information being obtained as long as all of the architectural components are properly managed, maintained and monitored. Although it employs a number of layers of security implemented through a variety of security measures, no system can provide absolute protection of your information. Only through constant vigilance can the system be properly secured.

Multiplication and Management are Key

Ultra-secure architecture relies on multiple network and application firewalls. This reduces the threat from application-based attacks such as injections, buffer overflows and other application-focused attacks often undetected or even handled by traditional network firewalls.

Also, the architecture uses two DMZs: one is available to the Internet (public) and the other is private. The servers in the public DMZ contain only application user-interface logic without any application processing logic. The servers in the private DMZ contain the actual application processing logic and links to internal systems for additional processing capabilities. Also, notice that the servers in the public DMZ are isolated from the systems with the application logic in the private DMZ. This allows the organization to make more defined rules for accessing the application logic so that application-based attacks do not work.

The ultra-secure architecture also uses two internal LANs: the internal LAN containing the employee-accessible servers and systems that do not store sensitive information and a secure LAN containing servers with encrypted information that could be used for identity theft or other frauds (credit card numbers, checking account numbers, check images, etc.). Finally, default ports for HTTP and HTTPS (tcp/80 and tcp/443) are used in the public DMZ and non-standard tcp and udp ports are used for all other connections to necessary services. This reduces the possibility of outside attackers accidentally identifying information assets through standard port injection attacks.

All components are maintained via a complete management and monitoring system implemented in a protected management LAN. This consists of intrusion detection/prevention system(s), Domain Name Services, Kerberos servers, time server(s) and system log (syslog) server(s). All of these servers are also firewalled from the DMZs and the secure LAN to allow for better control and protection. Users of your Web applications can process through the private DMZ or process through the public DMZ, depending on the applications.

Ultra-Secure Architecture Security Configuration

The following are the foundational architecture components for protecting the various systems, but the configuration, interaction and management of these components are what secure and monitor the architecture.

Intrusion-Detection System(s)
Ultra-secure architecture implements both network-based and host-based intrusion-detection system(s), and the key is implementing and properly managing and monitoring them. At a minimum, a network-based intrusion-detection system (NIDS) monitors all critical subnets in the DMZs and secure LAN. This will allow for the detection of any network-based attacks or unexpected network traffic anomalies. Additional NIDSs can be placed on other network segments, but this may result in significant amounts of tuning to minimize false positive alerts and other issues, since this network is not strictly controlled.

In addition to the NIDS, a host-based intrusion-detection system (HIDS) is implemented on all servers in the DMZs, all servers in the secure LAN and any servers that process sensitive information in the internal LAN. These HIDSs will detect file changes, brute force attacks or other attacks focused on a specific server. All of the NIDSs and HIDSs send information back to an intrusion-detection console system in the management LAN for tracking and monitoring.

Time Server
An often overlooked but important server is a time server that ensures the proper functioning and analysis of information stored in the Syslog server(s). Determining what time standard to use on your network is vital. For large, international organizations, all network infrastructure devices such as routers, switches, firewalls, servers, etc., usually use Universal Coordinated Time (UTC), which is the same as Greenwich Mean Time. A single, consistent time zone and time-keeping method for all devices becomes critical when diagnosing or identifying an attack that may be occurring against multiple devices in different parts of the network. Using UTC allows all
events to be shown in the same timeframe without having to perform time zone conversions.

System Log Server(s)
These often overlooked servers capture system log (syslog) information from all infrastructure devices such as firewalls, routers, switches, servers and other critical operation systems.

Syslog servers can be implemented in pairs; however, because these devices collect a large volume of information, coordinating that volume between two servers can create a problem. Therefore, organizations typically have only a 1U server attached to a large Network-Attached Storage (NAS) through a storage area network (SAN) device so that a large amount of storage is available. All critical devices must be transmitting their syslog information to the server for recording and further analysis. These critical systems should be logging as many successful and failed events as possible. Only then can a complete picture of events be maintained for analysis and diagnostic purposes.

The configuration of the network and application firewalls is critical. Rules must be configured to restrict and control both inbound and outbound communications. For example, the network firewall in the public DMZ would be configured for only inbound and outbound tcp 80 and 443 traffic to the respective IP addresses of the HTTP or HTTPS servers. All other ports and protocols would be closed, as they are not needed. Between the public DMZ and the private DMZ, in the example only port 62134 should be open to restrict communication to the IP addresses of the servers involved. Between the private DMZ and the internal LAN, only ports required to communicate to the various servers should be open and restricted to the specific IP addresses of those servers. Likewise, only the port necessary to gain access to the SQL server should be open and restricted to only that IP address for the firewall between the internal LAN and the secure LAN.

The application firewalls must be configured to correspond to the various Web-based applications being executed. Unlike their network firewall brethren, specific rule recommendations are difficult, if not impossible, to make for these devices because of the wide variety of application protocols and implementations. However, in general terms, all attacks that involve misuse of the various application protocols should be blocked.

Domain Name Service (DNS) Servers
These servers are purely for internal use only. Any external requests for DNS should be forwarded to your Internet service provider’s (ISP) DNS servers for resolution or further forwarding.

Because of the threats to DNS servers, it is always preferable to use your ISP’s DNS servers for your public DNS. While this can create timing issues with DNS changes, once an eCommerce system has been implemented and placed into production, DNS changes are typically rare.

Secure LAN Server(s) 
Servers located in the secure LAN provide their own level of security by storing only encrypted information. But this is just part of the story. The key to securing servers in the secure LAN is employing the “roach motel” concept: Information flows in, is encrypted and stored, but it does not flow out without an act of God.

In addition, an extremely limited number of system and network administration and application processes have access to these servers and the secure LAN. Application processes with access to these servers are restricted and monitored to ensure that only appropriate information flows are processed.

When approved, information is decrypted for processing needs. Those outflows are severely restricted, documented in detail, restricted by firewalls, and monitored and approved by management.

Kerberos Servers
These servers are the final key in creating an ultra-secure architecture. If you are not familiar with Kerberos, you should learn about it as soon as possible, as it will likely become vital to the overall security of your network. To work properly, Kerberos servers must be implemented in pairs for redundancy. According to the Massachusetts Institute of Technology (MIT) Kerberos Web site, Kerberos "uses strong cryptography so that a client can prove its identity to a server (and vice versa) across an insecure network connection. After a client and server have used Kerberos to prove their identity, they can also encrypt all of their communications to assure privacy and data integrity as they go about their business"

All servers used in the application process should use Kerberos to authenticate to one another. In addition, if possible, every individual application transaction session should generate its own Kerberos keys so that sessions are individually secured and encrypted.

Using Kerberos servers minimizes the possibility of man-in-the-middle attacks, packet sniffing and session hijacking, and it provides that extra level of security by encrypting all communications between systems. So, even if an attacker finds some way into the internal network, all communication is secured and encrypted.

Enhancements to the Ultra-Secure Architecture

A number of enhancements would increase the level of security given by the base-level ultra-secure application architecture. These enhancements, which add even more security layers to the basic ultra-secure architecture, could be implemented by companies with a greater concern for security.

Use of Multiple Subnets
The use of multiple IP subnets with the management LAN has already been referred to. However, multiple IP subnets could be used in each of the other four discrete networks as well. Each of the DMZs could have its own IP subnets that do not reflect any other internally used subnets (i.e., if the 10.x.x.x subnet is used internally and the 192.168.x.x subnet is used for the management LAN, the DMZs should use IP addresses in the 172.16-31.x.x subnet). The secure LAN should also have its own IP subnet that does not reflect any other internally used subnet.

While the use of multiple IP subnets will increase the amount of routing and the complexity of your network architecture, it also significantly reduces the likelihood that an attacker can readily gain access to systems and allows you to better implement and fine-tune your network monitoring activities.

The use of multiple IP subnets in conjunction with some of the other enhancements mentioned here will even further your security posture.

Use of Virtual Local Area Networks (VLAN)
VLANs logically segregate network traffic and can be used as part of your security program to segregate traffic based on the risk presented.

In the ultra secure architecture, VLANs can be created for operational control and for monitoring and segregating Internet-originated traffic and transactions from internal traffic. VLAN1, the default VLAN for most switches, is recommended for the operational control and monitoring function. Only network, system and security administration personnel should have access to this VLAN. VLAN1 should use only static IP addressing, which should not reflect the IP addressing scheme used by any other network. Since most servers come with dual network interface cards (NIC), one of these NICs can be configured to use only VLAN1. This allows for the server to be monitored and controlled over a secure network to which only network and system administration personnel have access.

Other VLANs can be established for Voice over IP (VoIP), ultra-secure architecture network traffic, internal LAN traffic, and secure wireless and unsecured wireless, among other things. The key to good VLAN implementations is to rigorously plan for your VLANs based on security, risk, network traffic, quality of service (Quest) and other considerations. However, a good VLAN plan leads to a flexible, reliable and secure networking environment.

VLANs have received a lot of press regarding the ways they can be circumvented through various attacks. All of these attacks are based on misconfigurations of the VLAN. Properly configured with the appropriate access control lists (ACL), VLANs can be a secure part of a network’s security posture.

Use of Virtual Machine (VM) Technology
This is a relatively new approach to improving your security posture, and it can provide some interesting possibilities.

The most widely know commercial VM software solution is VMware from EMC2, but Microsoft also offers a solution through VirtualPC. Other virtualization solutions are available for Linux platforms from Sourceforge and other open sources.

Briefly, VM introduces the ability to run multiple, logical, discrete operating systems (OS) on a single physical computer system or a cluster of computer systems. VMware can run on Windows or Linux hosts (GSX), or it can provide its own host environment for its Enterprise solution (ESX). Interestingly, some VMware users running in the Linux environment report that Windows VMs actually run faster than their native counterparts do on similar systems.

To enhance the ultra-secure architecture, VM would allow the HIDS to drop and reboot a server at will. If an HIDS indicates that a server has become corrupted or tampered with, the HIDS can be configured to automatically reboot the server. At the same time, a second hot swappable image can be running on the same physical system that takes over for the corrupted system. The rebooted system restarts from a known uncorrupted image and is then placed back in service. As a result, should an attacker corrupt the server, any rootkits or other unauthorized software is automatically wiped out because the server always reboots from a clean, known image.

Syslog/IDS/IPS Correlation Engine
The final potential enhancement to the ultra-secure architecture is the implementation of a syslog/IDS/IPS correlation engine application. As the name implies, these systems look for patterns in all syslog information and alert information (from IDS/IPS and SNMP information) that could indicate an attack or a network anomaly that should be investigated. Because of the complexity involved in setting appropriate rules for such correlation engines, this is not a task for the faint of heart and requires a significant amount of planning and monitoring of information to develop feasible rules. However, once implemented, these systems can significantly reduce the amount of alerts and other “chaff” so that network and security administration personnel can focus on more likely threats and anomalies versus all the possible alerts and anomalies that occur in a normal network.

Ultra-secure network architecture is unavoidably complex, as its complexity creates security. RSM’s technology risk management team has extensive experience in assessing, developing and maintaining sound internal controls for clients in a variety of industries, and can help your organization comply with ever-changing information security and privacy regulations.

How can we help you?

Contact us by phone 800.274.3978 or
submit your questions, comments, or proposal requests.

Rapid Assessment®

Complete our Rapid Assessment form to be contacted about receiving our "quick-hit" diagnostic of your critical areas of operations.




2017 cybersecurity outlook and key considerations for nonprofits

  • January 31, 2017


2017 economic and risk outlook

  • January 09, 2017


AML and regulatory compliance webcast series—Fall 2016

  • December 15, 2016


PCI DSS 3.2—What’s next?

  • December 08, 2016


RSM Raleigh Technology Conference

  • October 26, 2016