Deep Packet Inspection

Article Info

Contributed by
2 authors

Last updated on
2020-04-22 09:53:51

Improve this article

IP Networking
OSI Reference Model
Network Security
Denial of Service
Extended Berkeley Packet Filter
Data Analytics

Article Versions

4 2020-04-22 09:53:51
2021,2020 4,2021

By arvindpdmn

Minor formatting changes. References format edited. Image captions edited. Open source tools added.
3 2020-04-19 17:38:54
2020,1986 3,2020

By anuradhac

Updated discussion, milestones. Please review and publish.
2 2020-02-28 10:52:49
1986,1984 2,1986

By anuradhac

Added Summary and Discussion Qs. Pls check.
1 2020-02-25 12:59:05
1,1984

By anuradhac

First Draft

Chat Room

Submitting ...

You are editing an existing chat message.
2020-04-22 09:59:20
-

By arvindpdmn

Article just published. The image with bar graph is adapted from two slides. But the sources are different from these two slides, not just from summer 2013.
Image is not that relevant to performance. BitTorrent might impact other traffic types but I believe the question is more about how DPI impacts performance.
2020-03-05 04:01:44
-

By arvindpdmn

Good set of questions. The image in Summary is good.

A useful reference I found: https://digitalguardian.com/blog/what-deep-packet-inspection-how-it-works-use-cases-dpi-and-more

One additional question could be about tools to do DPI. The development of these tools could also come in the milestones. A useful reference where the author mentions a number of tools: https://security.ias.edu/deep-packet-inspection-dead-and-heres-why

Deep Packet Inspection. Source: Moxa 2020.

Traditionally, control and regulation of Internet traffic has been managed by a firewall in the router device. However, routers can only scan the header of an IP packet which contains source, destination addresses and some next-hop routing information.

Deep Packet Inspection is a technology that allows a service provider to analyse network traffic in real time using the payload (IP packet content), not merely the IP header. Packets are inspected based on rules assigned by an enterprise, government or internet service provider. Only packets which clear the inspection can enter the network. Even encrypted data can be analysed.

DPI can effectively monitor, speed up, slow down, block, filter, make decisions about the traffic. Mobile and broadband service providers widely employ DPI analysers in their networks. However, unless used judiciously, DPI can also result in invasion of data privacy and other internet governance issues.

Discussion

Why do we need Deep Packet Inspection?
Incoming Packet Filtering in DPI. Source: Cho and Mangione-Smith 2008, fig. 1.
DPI was initially intended to manage and safeguard Local Area Network users (such as universities, corporates) from malicious software or viruses. The idea was to intercept the malicious packets in real time, at a checkpoint before they reached end users. This was usually performed as a firewall feature.
DPI (also called Packet analysis) can be used both in Intrusion Detection Systems (IDS) and Intrusion Prevention Systems (IPS). In organizations which have remote users who connect using their laptops for work, DPI is vital in preventing worms, spyware, and viruses from getting into the corporate network.
DPI protects networks from spam, viruses, DDoS (Distributed Denial Of Service) attacks and harmful/illegal content. It also supports regulatory requirements for lawful intercept; and for parental or enterprise content control systems.
DPI allows governments and organizations to define their own rules and policies, so that the network can detect if there are prohibited uses of applications.
Another intended use is for network management such as ensuring a basic quality of service (QoS) for end-users and preventing network congestion due to trivial/spam content.
How does DPI work and what are its techniques?
Pattern Matching in DPI. Source: OpenPR 2018.
Traditional packet analysis tools only scanned packets at the IP and TCP layers, whereas DPI functions at the Application layer of the OSI reference model.
There are two different approaches to packet analysis - (1) Continual, full-scale traffic packet capture that requires high speed processing and large storage arrays, which is expensive. (2) On-demand packet capture only when system compatibility issues occur (missing / damaged packets).
DPI is actually a combination of several techniques:
- Flow Tracking: Determines which packets are part of a flow between the source and destination computers. It's based on a 5-tuple identifier (SRC-IP, DEST-IP, SRC-PORT, DEST-PORT, PROTOCOL).
- Pattern Matching: String patterns (several, not just one) coded as regular expressions are matched with incoming packets. For example, regardless of port, L7-filter classifier for Linux's Netfilter can classify packets as HTTP, Jabber, Citrix, Bittorrent, FTP, etc.
- Statistical Analysis: Indicators (mean, variance) on absolute / relative packet sizes, flow rate per application.
For network traffic monitoring, analytics from Gartner estimate that flow analysis should be done 80% of the time, and packet capture with probes should be done 20% of the time.
What's the architecture of DPI deployment in a network?
Inline Enabling of DPI. Source: Accolade 2020.
DPI engines are usually deployed inline with firewalls in routers, SDN, and packet gateways. Offline packet analysis can also be performed for non-critical analysis.
DPI is a standard option in 4G LTE and 5G packet gateways (P-GWs). For instance, the backbone network of an ISP could be a 40-Gb/s system with four 10-Gb/s DPI modules.
In the dynamic service environment implied by cloud/SDN, DPI could potentially be co-located with network devices (as software running in virtual switches) or in the control layer (in the controller between applications and switches) due to its high CPU resource requirement. Real-time analytics using DPI is fed into big data analytics packages. This helps service providers understand what end users are doing and shape service offerings accordingly.
Network performances are proved by practical tests executed using real traffic in an ISP backbone network. DPI devices are designed to handle thousands of transactions each second. These devices analyse less than 1500 bytes per packet, which is not a very heavy load. But if this happens in real time for every packet, there is network delay overhead.
What are the various other applications where DPI is used?
Interest in DPI technology. Source: Finnie 2012, fig. 4.
- Content Optimization: DPI can act as proxy and modify contents in order to reduce still/video image quality or reformat web pages for mobile devices as per available bandwidth/device constraints so that users can enjoy content with reasonable performance.
- Billing Applications: ISP use DPI to measure traffic volume and calibrate free and paid usages of network subscribers.
- Load Balancing: Redistribution of packet content to alternate servers in a load-balanced network to maintain uniform load across all deployed systems.
- User Behavior Analysis: Web/mobile applications can gauge subscriber behavior, assess what features are popular or which operations take too long, etc.
- Targeted Advertising: ISPs can inject advertisements into websites that match the assumed interests of the users according to their browsing habits.
- Copyright Enforcement: Automatically detect and block unauthorized sharing of music or video files on peer-to-peer platforms.
- Content Regulation: Recognize and block access to illegal/harmful content such as child-abuse websites. Censor content considered a threat to government or public stability.
If DPI analyses content in real time, will it not slow down the network data transmission?
Network Traffic statistics. Source: Mikov 2013
Many current DPI methods are resource intensive and costly, especially for high bandwidth applications. Since it's done in real time, it doesn't work on normal processors or switches.
DPI has become possible only in the last few years through advances in computer engineering and pattern matching algorithms. Specialized routers are now able to perform DPI. Routers armed with a dictionary of programs help identify the purposes behind the LAN and internet traffic they are routing. Vulnerability from repeat attacks from known viruses is removed. However, new viruses still pose a threat.
Interaction with legacy tools could also be a problem. Some firewalls simply aren't designed to support DPI, prompting worry about sudden performance drops or total failure of protective network systems.
In spite of these overheads, DPI is widely prevalent in network deployments because of the protection it offers from malicious or wasteful bandwidth usage by spams, viruses and malware. By successfully blocking such spurious input requests from untrustworthy clients, the DPI servers are able to save the network from unnecessary congestion and possible DDoS attacks.
What are the negative consequences of DPI and how can misuse be prevented?
DPI is basically a packet sniffing technology on the network traffic, enabling operators to monitor what is happening in real time. It was meant to be used for benevolent causes such as managing bandwidth, lawful surveillance, copyright enforcement, and network security.
However, DPI may be controversial from a customer privacy and net-neutrality standpoint. ISPs often use DPI servers to inspect Internet traffic to identify what traffic they want to slow down or restrict. At its root, DPI helps operators regain control over a network that primarily carries third-party applications and services by accurately identifying those applications in real time.
Privacy infringement, corporate snooping, governmental suppression of facts and news, and advertisement implantation are some of the negative consequences of DPI. If an ISP sells streaming music, then OTT music applications are its competitors. ISPs can intentionally ignore/cause congestion or degrade performance of the competing service.
One way to bypass DPI is to use traffic obfuscators as standalone software. These can change traffic signatures to look like traffic that isn't normally blocked by DPI.
What are the common tools used for packet analysis in DPI?
Governments of certain countries use proprietary and sophisticated DPI tools for online information censoring. The Great FireWall of China is one such example. They are implemented using a combination of flow analysers, filtering of certain IP ranges, flow redirection, URL filtering and Man-in-the-Middle techniques.
ntop, Netify Agent and libtins are open source utilities or toolkits in C/C++.
Corporates and Internet Service Providers may choose among the commonly available DPI tools:
- Wireshark: A popular free and open-source packet analyser which can be configured to be used for Intrusion Detection (ID). The tshark utility allows you to filter the contents of a pcap file from the command line to study network activity.
- Netfilter in Linux: Classifies packets as HTTP, Jabber, Citrix, Bittorrent, FTP, etc., regardless of port.
- Netflow from Cisco: Introduced on their routers to collect IP network traffic information as traffic enters/exits an interface and build Access Control Lists. It consists of a flow collector and analyser.
- SolarWinds Netflow: Network bandwidth monitoring (collection and analysis) tool. Free and paid versions available.
- Scrutinizer from Plixer: Can handle network flow analysis of Cisco and other vendors' network devices.

Milestones

1998

Wireshark, earlier part of the Ethereal project, is released as a free, open source packet sniffing tool. It initially supports shallow packet inspection, only at the IP header level.

2002

Traffic inspection solutions NetScreen (acquired by Juniper networks) are designed to be installed into firewall systems. Since the operation is expensive, it is triggered only on need basis.

2005

MIMESweeper, ClamAV, NetCache are some of the early open source internet proxy caching servers introduced for scanning content to an ICAP server running an anti-virus software.

2006

The Great Firewall of China is deployed successfully. This internet censorship project commenced in 1998 for online traffic regulation in China.

2012

DPI becomes a powerful network security tool with deployment on SDN/cloud servers.

2012

DPI analysis tools feed network traffic data into Big Data Analytics for ISPs to derive critical insights on user behavior.

References

Article Stats

1640

Words

Authors

Edits

Chats

Likes

9107

Hits

Cite As

Devopedia. 2020. "Deep Packet Inspection." Version 4, April 22. Accessed 2023-11-12. https://devopedia.org/deep-packet-inspection

Contributed by
2 authors

Last updated on
2020-04-22 09:53:51

Improve this article

web data networking protocol

IP Networking
OSI Reference Model
Network Security
Denial of Service
Extended Berkeley Packet Filter
Data Analytics

Deep Packet Inspection

Discussion

Milestones

References

Further Reading

Article Stats

Cite As

See Also

Deep Packet Inspection

Discussion

Milestones

References

Further Reading

Article Stats

Author-wise Stats for Article Edits

Cite As

See Also

Article Warnings

Login