Whispers & Screams
And Other Things

The Web By Proxy

I've been working on networks for decades and for as long as I can remember, network proxies have existed. I first came across the idea when I worked for IBM as an SNA programmer back in the late 90s but it's in more recent years that network proxies have taken on more importance. 

Continue reading
546 Hits
0 Comments

What on earth is making my home network so slow! (Part 1)

Let's face it, we've all been there. Sitting wondering why on earth a network connection that, up until 5 minutes ago had been working just fine was now all but useless. Less tech savvy individuals may just shrug their shoulders and try again later but anybody else is left wondering why. As a reader of this blog post that fact automatically places you in the latter category. So, to the problem. Could it be that somebody else in the house has started a large download? If that's the case its the easiest to solve just by asking around but the plethora of devices that are in our houses today make the job a lot more complex. For me it was a long forgotten mobile phone owned by my son, left on charge under the bed and set to auto update its code and apps that proved the final straw and drove me to come up with a solution to this problem.

Lets look at the problem in the round first of all. Homes nowadays usually have a router which connects off to the cable company or to the telephone line. These routers allow all of the devices in the house to connect to the net whether on the wireless or the wired side of life. Its not uncommon for a home network to support 10 to 20 devices not all of which will be known about by every other member of the household. Any one of these devices has the potential to bring the network to its knees for hours at an end by starting a large download. Of course the possibility also exists that somebody else on the outside has gained access to your network and it's important that this is not overlooked.

The first step in getting a handle on the situation will be to take control of your home router and secure it so that it cannot be manipulated by anybody else. Most home routers nowadays have a small, cut-down, webserver running on board which allows a management user to access the management web page. By using this web page clients can change all of the settings on the device. The page is usually accessible by both the wired and the wireless network. If you are using a Windows machine the easiest way to establish a connection to this page is to do the following:

    1. Click the pearl button and in the box which says "search programs and files" type cmd and press enter. This should bring up a window which looks like that shown on the right. Inside this window, type the command "ipconfig". The output should also resemble that shown on the right showing among other things, the address of the default gateway. Take a careful note of this address. (192.168.1.1 in this case)

 

    1. Open up a browser, type this default gateway address into the address bar and click enter. If your router is new or poorly configured you should now be looking at the control page for the device. If the device is configured properly you should now be looking at a login prompt page.

 

    1. Once logged in you will then be able to control the settings of the router.



This post is not written to be a guide for any specific router so I will keep any further instructions necessarily wide in scope.

The following bullets will link to posts that will be made available soon which examine the different aspects of this problem. Check back soon to see them when they become available.

    • Who is connected? Checking to understand which devices are connected to your router on WIFI and wired networks and establishing whether or not they should be.

 

    • What are they doing? Most routers show a basic table of transferred bandwidth as a part of their reporting. This can be used to examine the usage on your network and ascertain which devices are consuming most of the network.

 

    • Securing my router. As touched on previously, the router should be configured appropriately so that only those users whom you wish to have access are able to access both the network and the routers management page.

 

    • Customising the routers code. Home routers purchased off the shelf nowadays have woefully inadequate firmware that is frequently shown to be buggy at best and insecure at worst. Consider replacing this firmware with a fully customisable open source router such as dd-wrt or tomato.

 

    • Open source router management. (Wireshark and SNMP) Want to take the control of your home network to the max. Consider implementing network management, bandwidth management and device management.



I hope this post has proved informative as an intro to controlling your home network. Check back soon for further updates.

Continue reading
1272 Hits
2 Comments

Too Much Information - Hadoop and Big Data

hHadoop, a free, Java-based programming framework that makes it possible to run applications on systems with thousands of nodes involving thousands of terabytes, supports the processing of large amounts of data in a distributed computing environment and is part of the Apache project sponsored by the Apache Software Foundation. Its distributed file system facilitates rapid data transfer rates among nodes and allows the system to continue operating uninterrupted in case of a node failure. This approach lowers the risk of catastrophic system failure, even if a significant number of nodes become inoperative.

Hadoop was inspired by Google's MapReduce, a software framework in which an application is broken down into numerous small parts. Any of these parts (also called fragments or blocks) can be run on any node in the cluster. Doug Cutting, Hadoop's creator, named the framework after his child's stuffed toy elephant. The current Apache Hadoop ecosystem consists of the Hadoop kernel, MapReduce, the Hadoop distributed file system (HDFS) and a number of related projects such as Apache Hive, HBase and Zookeeper.

The Hadoop framework is used by major players including Google, Yahoo and IBM, largely for applications involving search engines and advertising. The preferred operating systems are Windows and Linux but Hadoop can also work with BSD and OS X.

The rapid proliferation of unstructured data is one of the driving forces of the new paradigm of big data analytics. According to one study, we are now producing as much data every 10 minutes as was created from the beginning of recorded time through the year 2003.1 The preponderance of data being created is of the unstructured variety -- up to about 90%, according to the IDC.

Big data is about being able to not just capture a wide variety of unstructured data, but to also capturing that data and combining it with other data to gain new insights that can be used in many ways to improve business performance. For Instance, in retail, it could mean delivering faster and better services to customers; in research, it could mean conducting tests over much wider sampling sizes; in healthcare, it could mean faster and more accurate diagnoses of illnesses.

The ways in which big data will change our lives is significant, and just beginning to reveal itself for those who are willing to capture, combine, and discover answers to their Big Questions. For big data to deliver on the promise of its vast potential, however, technology must be in place to enable organizations to capture and store massive amounts of unstructured data in its native format. That’s where Hadoop has become one of the enabling data processing technologies for big data analytics. Hadoop allows for dramatically bigger business questions to be answered, that we are already starting to see realized from large public cloud companies, which will shortly infiltrate into other IT oriented industries and services.

More than 50% of participating companies have begun implementing the available Hadoop frameworks as data hubs or auxiliary data repositories to their existing infrastructures, according to Intel’s 2013 IT Manager’s Survey on How Organizations are Using Big Data. In addition, 31% more organizations reported evaluating one of open-source Apache Hadoop framework.

So what are the key characteristics IT professionals should know about Hadoop in order to maximize its potential in managing unstructured data and advancing the cause of big data analytics? Here are five to keep in mind:

    1. Hadoop is economical. As an open-source software framework, Hadoop runs on standard servers. Hardware can be added or swapped in or out of a cluster, and operational costs are relatively low because the software is common across the infrastructure, requiring little tuning for each physical server.

 

    1. Hadoop provides an efficient framework for processing large sets of data. MapReduce is the software programming framework in the Hadoop stack. Simply put, rather than moving data across a network to be processed, MapReduce provides a framework to move the processing software to the data.3 In addition to simplifying the processing of big data sets, MapReduce also provides programmers with a common method of defining and orchestrating complex processing tasks across clusters of computers.

 

    1. Hadoop supports your existing database and analytics infrastructures, and does not displace it. Hadoop can handle data sets and tasks that can be a problem for legacy databases. In big data environments, you want to make sure that the underlying storage and infrastructure platform for the database is capable of handling the capacity and speed of big data initiatives, particularly for mission-critical applications. Because of this capacity it can and has been implemented as a replacement to existing infrastructures, but only where it fits the business need or advantage

 

    1. Hadoop will provide the best value where it is implemented with the right infrastructure. The Hadoop framework typically runs on mainstream standard servers using common Intel® server hardware. Newer servers with the latest Intel® computing, larger memory footprint, and more cache will typically provide better performance. In addition, Hadoop will perform better with faster in node storage, so systems should contain some amount of solid-state storage. In addition, the storage infrastructure should be optimized with the latest advances in automated tiering, deduplication, compression, encryption, erasure coding and thin provisioning. When Hadoop has scaled to encompass larger datasets it benefits from faster networks, so then 10Gb Ethernet rather than typical 1GbE bandwidth provides further benefit.

 

    1. Hadoop is supported by a large and active ecosystem. Big data is a big opportunity, not just for those using it to deliver competitive advantage, but also to those providing solutions. A large and active ecosystem has developed quickly around Hadoop, as it usually does around open-source solutions. As an example, Intel recently invested $740 million dollars into the leading distribution for Hadoop provided by Cloudera. Vendors are available to provide all or part of the Hadoop stack, including management software, third-party applications and a wide range of other tools to help simplify the deployment of Hadoop.



Unstructured data is growing nonstop across a variety of applications, in a wide range of formats. Those companies that are best able to harness it and use it for competitive advantage are seeing significant results and benefits. That’s why more than 80% of the companies surveyed by Intel are using, implementing or evaluating Hadoop.

Continue reading
326 Hits
0 Comments

Network Functions Virtualization on the Software-Defined Network

banner_inter_urbanIn the modern Telecom industry, driven by the fast changing demands that a connected society makes of it, a huge number of new applications are emerging such as IPX, eHealth, Smart Cities and the Internet of Things. Each of these emergent applications requires new customisations of the ecosystem to manage traffic through a wide variety of service providers.

This is the core challenge faced by todays infrastructure, but we must also not overlook the fact that to serve this larger ecosystem requires an enormous change to OSS infrastructure and the way networks are being managed. Service providers are placed in the awkward space between the end users and the emergent technologies but it is the fact that these technologies and their business models are often emerging on a month to month basis that presents the greatest challenge.

If we consider all the IT assets ISP's and Telcos have at their Points of Presence it represents a significant and very much underused resource. The holy grail for many of these organisation is to be able to unlock all of this storage and computing capacity, and turn it into a virtualized resources. This strategy opens up some intriguing possibilities such as bringing remote resource to bear during times of heavy compute load at a specific locale from areas where capacity is less constrained. In infrastructure terms, this cloud-oriented world of adding new network capacity whenever and wherever it is needed is a matter of merely sliding more cards into racks or deploying new software which greatly lowers the cost of scaling the network hardware by commoditising the components used to build up a service providers infrastructure.

Agility of services is the key to this new world order where services can be created orders of magnitude more quickly than was traditionally the case. In this new model the division between content providers and service providers becomes blurred. The flexibility to manage this dynamism is the key to the industry being able to meet the demands that the connected society will increasingly place on it and it will be those players who are able to manage this balancing act most effectively that will come out on top.

This is where NFV comes in. The advent of Network Function Virtualization, or NFV, has strong parallels to those developments in the computing world that gave us the cloud, big data and other commodity computing advances. Using capacity where and when it is required with a lot less visibility into the physical location of the network than is needed currently presents a whole new set of unique challenges. As computing hardware has developed and become more capable, a greater level of software complexity has taken place by its side.

The management of NFV will be critical to its operation, and the way that end user functionality is moving to the cloud today represents a sneak preview of this. We’re seeing a preview of that as computing scales to the cloud. A lot of careful design consideration will be required and service providers need to begin adapting their infrastructure today to accommodate this future virtualization.

Closely related and indeed an enabler to the trend of NFV is the Software-Defined Network, or SDN. The SDN, or Software Defined Networking can provide improved network efficiency and better cost savings, allowing the network to follow-the-sun, turning down servers or network hardware when the load lightens, or even turning them off at night. In a wireless environment, for example, if you could turn off all the excess network capability not in use from 10 p.m. to 6 a.m., you will see a significant decrease in the cost of electricity and cooling.

The continued integration of technologies such as Openflow into the latest and greatest network management implementations will further enable this trend as we increasingly see these OSS and BSS systems seek to pre-empt their traditional reactive mechanisms by looking farther up the business model in order to steal vital time with which to maximise the effectiveness of their influence and ultimately maximise the value add of their managed virtualised domains.

Continue reading
329 Hits
0 Comments

Classful IP Addressing (IPv4)

cisco-ccna-subnetting-02IP addressing is among the most important topics in any examination of TCP/IP. The IP address is a 32 bit binary identifier which, when configured correctly, enables each machine on an IP network to be uniquely identified. It is used to allow communication with any specific device on the network.

An IP address is defined in software and is configured dynamically as needed by software whether controlled by a human or a software process (as opposed to a MAC address which is a permanent, hard coded hardware address which cannot be easily changed). IP addressing was designed to allow media independent communication between any two hosts on the same, or different, IP networks.

Terminology

As a precursor to looking at IP Addressing in some detail, lets define some basic terminology.

Byte - A byte is a unit of binary information in that most commonly consists of eight bits. In the course of this post, the term Octet will also be used to represent one and the same thing.

IP Address - An IP address is a 32 bit binary number which represents, when assigned to a network device, its unique Network Layer (Layer 3) address. IP addresses are commonly described in Dotted Decimal notation for ease of human readability. Dotted Decimal notation is the conventional way of describing an IP address eg. 192.168.1.1 and is formed by separating the 32 bit IP address into 4 x 8-bit Octets, converting each Octet into a decimal number between 0 and 255 and separating each of these Octets with a dot. An IP Address is also frequently referred to as a Network Address and the terms can be used interchangeably however IP Address is by far the most common.

Broadcast Address - On any IP Network, the Broadcast Address is the address used to send to all hosts which are members of and connected to the IP Network.

IP Addressing

As mentioned previously, an IP address is made up of 32 binary bits. It is extremely important to always bear this fact in mind when working with IP addresses as failing to do so can significantly impair ones ability to fully understand and manipulate the IP addressing system as required.
IP addresses are commonly described in one of three ways -

    1. Dotted Decimal (As described above)

 

    1. Binary (As a 32 bit binary number)

 

    1. Hexadecimal (Rarely used but can be seen when addresses are stored within programs or during packet analysis)



One important aspect of an IP address is that it is an hierarchical address. This has a number of advantages not least of which is the fact that it enables addresses to be aggregated together which greatly simplifies the mechanisms which are used to route traffic around the Internet.
In IPv4 there are 4.3 billion IP addresses available in theory and without this mechanism for route aggregation it would be necessary for Internet routers to know the location of each one of these connected devices.
The hierarchical system used by IPv4 is one which separates the IP address into two components, namely a network part and a host part.
In practice this "two component" system is further split down as the host part is frequently subdivided into even smaller subnetworks. In this post however we will limit or discussion to the "two component" system.

This term, "subnetwork", (often abbreviated to subnet) is one which is used frequently within the network engineering community to such an extent that it has become part of the jargon of the trade. This has only served to enhance its status as a term which has a great deal of complexity behind it but it is actually extremely simple. A subnetwork (subnet) is any subdivision of a larger network. It really is as simple as that.
The Two Component System / Network part and Host part

In order to make IP addresses hierarchical, a two component system has been created. This system splits the IP address into two parts known as The Network Part and The Host Part. This can be likened to a telephone number where (typically) the first 4 or 5 digits represent the town or city and the subsequent 6 or 7 digits represent the individual line.
The designers of this hierarchical addressing scheme created 5 classes of IP address by splitting up the full range of 4.3 billion addresses in a logical way. These 5 classes (or subdivisions) are known as Class A, B, C, D, and E networks.
For the purposes of this post we shall concern ourselves primarily with classes A, B and C however I shall briefly introduce each of the classes in the following section.
The 5 Network Classes
The image below depicts the 5 classes of IP Network as well as some of the basic features associated with each.

Screenshot_2

Class A - Class A networks were designed for use in networks which needed to accommodate a very large number of hosts.
As can be seen from the diagram, the first bit in a Class A address is always 0.
In each of network classes A, B and C, we can also see that the addresses are split into two parts, namely Network and Hosts.
These parts can be likened to the two parts of the telephone number described earlier.
The Network part is like the city code and the Host part is like the rest of the telephone number.
As you can see from the image, the division between the Network and Host part is set after the 8th bit. This means that we have 7 bits available to represent different Networks and 24 bits available to represent the individual hosts within each of the Class A networks.
It is clear therefore that, since the first bit must always be 0, the lowest network address available is 00000000.X.X.X (0 in decimal) and the highest network address available is 01111111.X.X.X (127 in decimal).
It would seem therefore that the range of addresses available to Class A networks is 0.X.X.X up to 127.X.X.X (Where X represents the Host part) but I shall demonstrate later that the 0 and 127 networks are reserved therefore the Class A address range runs from 1.X.X.X to 126.X.X.X in practice.

Class B - In Class B networks, the split between the network part and the host part happens after the 16th bit.
In any Class B network address the first two bits must always be set to 10. This leaves 14 bits to define the network number and allows addresses to range from 10000000.00000000.X.X up to 10111111.11111111.X.X .
These binary addresses equate in decimal to the first two Octets of Class B addresses ranging from 128.0.X.X up to 191.255.X.X
Class C - The pattern now emerging is that Class C addresses use the first 3 Octets to define the Network part of their addresses. Again, as with Class A and B networks some bits are permanently defined and in the case of Class C network addresses, the first 3 bits are always set to 110.
This means that we have 21 bits available to define the network part of Class C network addresses ranging (in binary) from 11000000.00000000.00000000.X up to 11011111.11111111.11111111.X which in decimal equates to 192.0.0.X up to 223.255.255.X

Class D - Class D (224-239) is reserved for Multicast Addressing and a post based explicitly on this addressing will be published ASAP and linked to from here. Class D addressing is beyond the scope of this post however if required please click this link for more detail. Class D Networks and IP Multicasting. .

Class E - Class E (240-255) is reserved for scientific experimentation and research and if any subsequent posts on this blog examine Class E networks, they will be linked to from here.

Continue reading
322 Hits
0 Comments