• Deep Packet Inspection. Source: Moxa 2020.
    Deep Packet Inspection. Source: Moxa 2020.
  • Incoming Packet Filtering in DPI. Source: Cho and Mangione-Smith 2008, fig. 1.
    Incoming Packet Filtering in DPI. Source: Cho and Mangione-Smith 2008, fig. 1.
  • Pattern Matching in DPI. Source: OpenPR 2018.
    Pattern Matching in DPI. Source: OpenPR 2018.
  • Inline Enabling of DPI. Source: Accolade 2020.
    Inline Enabling of DPI. Source: Accolade 2020.
  • Interest in DPI technology. Source: Finnie 2012, fig. 4.
    Interest in DPI technology. Source: Finnie 2012, fig. 4.
  • Network Traffic statistics. Source: Mikov 2013
    Network Traffic statistics. Source: Mikov 2013

Deep Packet Inspection

User avatar
anuradhac
1071 DevCoins
Avatar of user arvindpdmn
arvindpdmn
163 DevCoins
2 authors have contributed to this article
Last updated by arvindpdmn
on 2020-04-22 09:53:51
Created by anuradhac
on 2020-02-25 12:59:05

Summary

Deep Packet Inspection. Source: Moxa 2020.
Deep Packet Inspection. Source: Moxa 2020.

Traditionally, control and regulation of Internet traffic has been managed by a firewall in the router device. However, routers can only scan the header of an IP packet which contains source, destination addresses and some next-hop routing information.

Deep Packet Inspection is a technology that allows a service provider to analyse network traffic in real time using the payload (IP packet content), not merely the IP header. Packets are inspected based on rules assigned by an enterprise, government or internet service provider. Only packets which clear the inspection can enter the network. Even encrypted data can be analysed.

DPI can effectively monitor, speed up, slow down, block, filter, make decisions about the traffic. Mobile and broadband service providers widely employ DPI analysers in their networks. However, unless used judiciously, DPI can also result in invasion of data privacy and other internet governance issues.

Milestones

1998

Wireshark, earlier part of the Ethereal project, is released as a free, open source packet sniffing tool. It initially supports shallow packet inspection, only at the IP header level.

2002

Traffic inspection solutions NetScreen (acquired by Juniper networks) are designed to be installed into firewall systems. Since the operation is expensive, it is triggered only on need basis.

2005

MIMESweeper, ClamAV, NetCache are some of the early open source internet proxy caching servers introduced for scanning content to an ICAP server running an anti-virus software.

2006

The Great Firewall of China is deployed successfully. This internet censorship project commenced in 1998 for online traffic regulation in China.

2012

DPI becomes a powerful network security tool with deployment on SDN/cloud servers.

2012

DPI analysis tools feed network traffic data into Big Data Analytics for ISPs to derive critical insights on user behavior.

Discussion

  • Why do we need Deep Packet Inspection?
    Incoming Packet Filtering in DPI. Source: Cho and Mangione-Smith 2008, fig. 1.
    Incoming Packet Filtering in DPI. Source: Cho and Mangione-Smith 2008, fig. 1.

    DPI was initially intended to manage and safeguard Local Area Network users (such as universities, corporates) from malicious software or viruses. The idea was to intercept the malicious packets in real time, at a checkpoint before they reached end users. This was usually performed as a firewall feature.

    DPI (also called Packet analysis) can be used both in Intrusion Detection Systems (IDS) and Intrusion Prevention Systems (IPS). In organizations which have remote users who connect using their laptops for work, DPI is vital in preventing worms, spyware, and viruses from getting into the corporate network.

    DPI protects networks from spam, viruses, DDoS (Distributed Denial Of Service) attacks and harmful/illegal content. It also supports regulatory requirements for lawful intercept; and for parental or enterprise content control systems.

    DPI allows governments and organizations to define their own rules and policies, so that the network can detect if there are prohibited uses of applications.

    Another intended use is for network management such as ensuring a basic quality of service (QoS) for end-users and preventing network congestion due to trivial/spam content.

  • How does DPI work and what are its techniques?
    Pattern Matching in DPI. Source: OpenPR 2018.
    Pattern Matching in DPI. Source: OpenPR 2018.

    Traditional packet analysis tools only scanned packets at the IP and TCP layers, whereas DPI functions at the Application layer of the OSI reference model.

    There are two different approaches to packet analysis - (1) Continual, full-scale traffic packet capture that requires high speed processing and large storage arrays, which is expensive. (2) On-demand packet capture only when system compatibility issues occur (missing / damaged packets).

    DPI is actually a combination of several techniques:

    • Flow Tracking: Determines which packets are part of a flow between the source and destination computers. It's based on a 5-tuple identifier (SRC-IP, DEST-IP, SRC-PORT, DEST-PORT, PROTOCOL).
    • Pattern Matching: String patterns (several, not just one) coded as regular expressions are matched with incoming packets. For example, regardless of port, L7-filter classifier for Linux's Netfilter can classify packets as HTTP, Jabber, Citrix, Bittorrent, FTP, etc.
    • Statistical Analysis: Indicators (mean, variance) on absolute / relative packet sizes, flow rate per application.

    For network traffic monitoring, analytics from Gartner estimate that flow analysis should be done 80% of the time, and packet capture with probes should be done 20% of the time.

  • What's the architecture of DPI deployment in a network?
    Inline Enabling of DPI. Source: Accolade 2020.
    Inline Enabling of DPI. Source: Accolade 2020.

    DPI engines are usually deployed inline with firewalls in routers, SDN, and packet gateways. Offline packet analysis can also be performed for non-critical analysis.

    DPI is a standard option in 4G LTE and 5G packet gateways (P-GWs). For instance, the backbone network of an ISP could be a 40-Gb/s system with four 10-Gb/s DPI modules.

    In the dynamic service environment implied by cloud/SDN, DPI could potentially be co-located with network devices (as software running in virtual switches) or in the control layer (in the controller between applications and switches) due to its high CPU resource requirement. Real-time analytics using DPI is fed into big data analytics packages. This helps service providers understand what end users are doing and shape service offerings accordingly.

    Network performances are proved by practical tests executed using real traffic in an ISP backbone network. DPI devices are designed to handle thousands of transactions each second. These devices analyse less than 1500 bytes per packet, which is not a very heavy load. But if this happens in real time for every packet, there is network delay overhead.

  • What are the various other applications where DPI is used?
    Interest in DPI technology. Source: Finnie 2012, fig. 4.
    Interest in DPI technology. Source: Finnie 2012, fig. 4.
    • Content Optimization: DPI can act as proxy and modify contents in order to reduce still/video image quality or reformat web pages for mobile devices as per available bandwidth/device constraints so that users can enjoy content with reasonable performance.
    • Billing Applications: ISP use DPI to measure traffic volume and calibrate free and paid usages of network subscribers.
    • Load Balancing: Redistribution of packet content to alternate servers in a load-balanced network to maintain uniform load across all deployed systems.
    • User Behavior Analysis: Web/mobile applications can gauge subscriber behavior, assess what features are popular or which operations take too long, etc.
    • Targeted Advertising: ISPs can inject advertisements into websites that match the assumed interests of the users according to their browsing habits.
    • Copyright Enforcement: Automatically detect and block unauthorized sharing of music or video files on peer-to-peer platforms.
    • Content Regulation: Recognize and block access to illegal/harmful content such as child-abuse websites. Censor content considered a threat to government or public stability.
  • If DPI analyses content in real time, will it not slow down the network data transmission?
    Network Traffic statistics. Source: Mikov 2013
    Network Traffic statistics. Source: Mikov 2013

    Many current DPI methods are resource intensive and costly, especially for high bandwidth applications. Since it's done in real time, it doesn't work on normal processors or switches.

    DPI has become possible only in the last few years through advances in computer engineering and pattern matching algorithms. Specialized routers are now able to perform DPI. Routers armed with a dictionary of programs help identify the purposes behind the LAN and internet traffic they are routing. Vulnerability from repeat attacks from known viruses is removed. However, new viruses still pose a threat.

    Interaction with legacy tools could also be a problem. Some firewalls simply aren't designed to support DPI, prompting worry about sudden performance drops or total failure of protective network systems.

    In spite of these overheads, DPI is widely prevalent in network deployments because of the protection it offers from malicious or wasteful bandwidth usage by spams, viruses and malware. By successfully blocking such spurious input requests from untrustworthy clients, the DPI servers are able to save the network from unnecessary congestion and possible DDoS attacks.

  • What are the negative consequences of DPI and how can misuse be prevented?

    DPI is basically a packet sniffing technology on the network traffic, enabling operators to monitor what is happening in real time. It was meant to be used for benevolent causes such as managing bandwidth, lawful surveillance, copyright enforcement, and network security.

    However, DPI may be controversial from a customer privacy and net-neutrality standpoint. ISPs often use DPI servers to inspect Internet traffic to identify what traffic they want to slow down or restrict. At its root, DPI helps operators regain control over a network that primarily carries third-party applications and services by accurately identifying those applications in real time.

    Privacy infringement, corporate snooping, governmental suppression of facts and news, and advertisement implantation are some of the negative consequences of DPI. If an ISP sells streaming music, then OTT music applications are its competitors. ISPs can intentionally ignore/cause congestion or degrade performance of the competing service.

    One way to bypass DPI is to use traffic obfuscators as standalone software. These can change traffic signatures to look like traffic that isn't normally blocked by DPI.

  • What are the common tools used for packet analysis in DPI?

    Governments of certain countries use proprietary and sophisticated DPI tools for online information censoring. The Great FireWall of China is one such example. They are implemented using a combination of flow analysers, filtering of certain IP ranges, flow redirection, URL filtering and Man-in-the-Middle techniques.

    ntop, Netify Agent and libtins are open source utilities or toolkits in C/C++.

    Corporates and Internet Service Providers may choose among the commonly available DPI tools:

    • Wireshark: A popular free and open-source packet analyser which can be configured to be used for Intrusion Detection (ID). The tshark utility allows you to filter the contents of a pcap file from the command line to study network activity.
    • Netfilter in Linux: Classifies packets as HTTP, Jabber, Citrix, Bittorrent, FTP, etc., regardless of port.
    • Netflow from Cisco: Introduced on their routers to collect IP network traffic information as traffic enters/exits an interface and build Access Control Lists. It consists of a flow collector and analyser.
    • SolarWinds Netflow: Network bandwidth monitoring (collection and analysis) tool. Free and paid versions available.
    • Scrutinizer from Plixer: Can handle network flow analysis of Cisco and other vendors' network devices.

References

  1. Accolade. 2020. "In-Line Enabling of DPI." Accessed 2020-04-14.
  2. Bebop, Cowbear. 2017. "GhostBear: How to Hide Your VPN From DPI." Blog, Tunnel Bear, July 18. Accessed 2020-04-14.
  3. Brook, Chris. 2018. "What is Deep Packet Inspection? How It Works, Use Cases for DPI, and More." Digital Guardian, December 5. Accessed 2020-04-14.
  4. Byres, Eric. 2013. "Understanding Deep Packet Inspection for SCADA Security of Offshore Production Facilities." Tofino Security, Belden. Accessed 2020-02-28.
  5. Cho, Young Hyun and William H. Mangione-Smith. 2008. "Deep network packet filter design for reconfigurable devices." ACM Transactions on Embedded Computing Systems, vol. 7, no. 2, pp. 21:1-21:26, February. Accessed 2020-04-14.
  6. Cignoli, Christine. 2018. "Avoiding the Pitfalls of Purpose: The Case for Deep Packet Inspection." Blog, AppNeta, January 23. Accessed 2020-04-14.
  7. Connet. 2014. "Deep Packet Inspection" Connet, Inc. Accessed 2020-04-14.
  8. Finnie, Graham. 2012. "White Paper - The Role of DPI in an SDN World." December. Qosmos. Accessed 2020-04-14.
  9. Glass, Charles. 2016. "DPI in an SDN world." SlidePlayer. Accessed 2020-04-14.
  10. Kim, NamUk and J. Choi. 2014. "A Scalable Carrier-Grade DPI System Architecture Using Synchronization of Flow Information." IEEE Journal on Selected Areas in Communications, vol. 32, no. 10, pp. 1834-1848, October. Accessed 2020-04-14.
  11. Larue-Langlois, Renaud. 2018. "5 Best Tools For Deep Packet Inspection in 2020." AddictiveTips, November 27. Accessed 2020-04-14.
  12. Mikov, Ilya. 2013. "DPI BOX: deep packet inspection for ISP traffic management." DPI-BOX, via SlideShare, August 12. Accessed 2020-04-14.
  13. Minarik, Pavel. 2019. "How to analyze and understand your network." Flowmon Networks. Accessed 2020-04-14.
  14. Moxa. 2020. "How Deep Packet Inspection Helps Protect Industrial Control Systems." Moxa Inc. Accessed 2020-02-28.
  15. NetScreen. 2018. "NetScreen Security Products Hardware Dates & Milestones." Juniper Networks, Inc. Accessed 2020-04-14.
  16. Nolle, Tom. 2009. "Deep packet inspection: Controversial but valuable traffic management tool." TechTarget, July. Accessed 2020-04-14.
  17. OpenPR. 2018. "Deep Packet Inspection And Processing Market Growth Augmenting Due to Increasing Use Of Mobile Devices With Growing Mobile Broadband Data Demand." April 28. Accessed 2020-04-14.
  18. Saurabh, Aman. 2017. "A Guide to Deep Packet Inspection." Catchpoint Systems, Inc., July 19. Accessed 2020-04-14.
  19. Schulze, Hendrik. 2012. "Part 2: Deep Packet Inspection Tutorial." December 19. Accessed 2020-04-14.
  20. Sinha, Amber. 2016. "Deep Packet Inspection: How it Works and its Impact on Privacy." Center for Internet & Society, December 16. Accessed 2020-02-28.
  21. Sommer, Ethan and Mathew Strait. 2009. "Application Layer Packet Classifier for Linux." January 7. Accessed 2020-04-14.
  22. TEC. 2019. "Whitepaper on Deep Packet Inspection." September 20. Accessed 2020-02-28.
  23. VyprVPN. 2015. "Your ISP is Slowing you Down. Defeat Throttling & Peering with a VPN." Blog, VyprVPN, Golden Frog, GmbH, September 24. Accessed 2020-04-14.
  24. Wikipedia. 2019. "Deep Content Inspection." Wikipedia, November 25. Accessed 2020-04-14.
  25. Wikipedia. 2020. "Great Firewall." Wikipedia, April 15. Accessed 2020-04-14.
  26. Wireshark. 2009. "Deep packet inspection with Wireshark." Packet Storm. Accessed 2020-04-14.

Milestones

1998

Wireshark, earlier part of the Ethereal project, is released as a free, open source packet sniffing tool. It initially supports shallow packet inspection, only at the IP header level.

2002

Traffic inspection solutions NetScreen (acquired by Juniper networks) are designed to be installed into firewall systems. Since the operation is expensive, it is triggered only on need basis.

2005

MIMESweeper, ClamAV, NetCache are some of the early open source internet proxy caching servers introduced for scanning content to an ICAP server running an anti-virus software.

2006

The Great Firewall of China is deployed successfully. This internet censorship project commenced in 1998 for online traffic regulation in China.

2012

DPI becomes a powerful network security tool with deployment on SDN/cloud servers.

2012

DPI analysis tools feed network traffic data into Big Data Analytics for ISPs to derive critical insights on user behavior.

Tags

See Also

  • IP Networking
  • OSI Reference Model
  • Network Security
  • Denial of Service
  • Extended Berkeley Packet Filter
  • Data Analytics

Further Reading

  1. Brook, Chris. 2018. "What is Deep Packet Inspection? How It Works, Use Cases for DPI, and More." Digital Guardian, December 5. Accessed 2020-04-14.
  2. Wilson, Marc. 2019. "Deep Packet Inspection – A Look at What It Is, Tutorial & Software/Tools for DPI." PCWDLD.com, March 30. Accessed 2020-04-22.
  3. Cooper, Stephen. 2018. "Deep Packet Inspection (DPI) Guide Including 7 Best DPI Tools." Comparitech, March 1. Accessed 2020-04-22.

Article Stats

Author-wise Stats for Article Edits

Author
No. of Edits
No. of Chats
DevCoins
3
0
1071
1
2
163
1640
Words
2
Chats
4
Edits
0
Likes
865
Hits

Cite As

Devopedia. 2020. "Deep Packet Inspection." Version 4, April 22. Accessed 2020-08-12. https://devopedia.org/deep-packet-inspection