{"id":1253,"date":"2013-01-30T00:00:09","date_gmt":"2013-01-29T23:00:09","guid":{"rendered":"http:\/\/cerezo.name\/blog\/?p=1253"},"modified":"2024-10-14T13:41:42","modified_gmt":"2024-10-14T11:41:42","slug":"reverse-engineering-network-protocols","status":"publish","type":"post","link":"https:\/\/cerezo.name\/blog\/2013\/01\/30\/reverse-engineering-network-protocols\/","title":{"rendered":"Reverse Engineering Network Protocols"},"content":{"rendered":"<p style=\"text-align: justify;\">The black arts of reverse engineering network protocols have been lost. These days, every network protocol seems to be run over <span class=\"caps\">HTTP<\/span> and handling lots of <span class=\"caps\">XML<\/span>: every network engineer of the past decades would just cringe at the thought of&nbsp;it.<\/p>\n<p style=\"text-align: justify;\">Complete specifications of network protocols like those offered in RFCs have always been luxuries: the product of idealistic minds of the past like Jon Postel, they only exist for the better known protocols of the Internet. For the rest, their details could only be known by reverse engineering: and the truth is that it requires a deep understanding of traditional software debugging, using tools like <a href=\"https:\/\/www.hex-rays.com\/products\/ida\/index.shtml\" target=\"_blank\" rel=\"noopener\"><span class=\"caps\">IDA<\/span><\/a>&nbsp;and\/or <a href=\"http:\/\/www.ollydbg.de\" target=\"_blank\" rel=\"noopener\">OllyDbg<\/a>, specially for protocols of the binary kind.<\/p>\n<p style=\"text-align: justify;\">Thus, the case of Skype: a <a href=\"http:\/\/skype-open-source.blogspot.com\" target=\"_blank\" rel=\"noopener\" class=\"broken_link\">recent decompilation<\/a> of its binaries using <a href=\"https:\/\/www.hex-rays.com\/products\/decompiler\/index.shtml\" target=\"_blank\" rel=\"noopener\">Hex-Rays<\/a> was publicly sold as a reverse engineering of the whole protocol suite. Nothing could be further from the&nbsp;truth.<\/p>\n<p style=\"text-align: justify;\">Providing yourself with a kit of the best tools is the best path to success:<\/p>\n<ul style=\"text-align: justify;\">\n<li>Sniffers are boring, read-only tools to see through the network layers. More fun can be had by crafting network packets, as recently simplified by tools like <a href=\"http:\/\/code.google.com\/p\/ostinato\" target=\"_blank\" rel=\"noopener\">Ostinato<\/a>&nbsp;and <a href=\"http:\/\/www.secdev.org\/projects\/scapy\" class=\"broken_link\">scapy<\/a><\/li>\n<li>Another set of tools focus on decoding text-like protocols: <a href=\"http:\/\/code.google.com\/p\/reverx\" target=\"_blank\" rel=\"noopener\">reverx<\/a>&nbsp; (<a href=\"http:\/\/www.di.fc.ul.pt\/~nuno\/PAPERS\/WCRE11.pdf\" target=\"_blank\" rel=\"noopener\">paper<\/a>), and the impressive <a href=\"http:\/\/www.netzob.org\">netzob<\/a><\/li>\n<li>And the more interesting ones, tools that cross-overs between debuggers and sniffers: <a href=\"http:\/\/code.google.com\/p\/ospy\" target=\"_blank\" rel=\"noopener\">oSpy<\/a>, an utility to sniff network application calls, and <a href=\"http:\/\/code.google.com\/p\/windbgshark\" target=\"_blank\" rel=\"noopener\">windbgshark<\/a>, an extension to integrate wireshark within windbg to manipulate virtual machine network traffic<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">It\u2019s said that in computer science, there\u2019s only a sure way to find a research topic to write papers about: just add <em>automatic<\/em> to any problem statement, and a whole area of research is born! (aka. the <em>meta-folk theorem<\/em> of <span class=\"caps\">CS<\/span> research). Most of the time the topic is obviously undecidable and a huge effort will be needed to produce tools of real practical value, but this doesn\u2019t seem to stop researchers to produce interesting Proof-Of-Concepts. Reverse engineering being such a painstaking manual process, it\u2019s a perfect target for this way of producing research, and very different methods and approaches have been tested: Smith-Waterman and Needleman-Wunsch <a href=\"http:\/\/www.4tphi.net\/~awalters\/PI\/pi.pdf\" target=\"_blank\" rel=\"noopener\" class=\"broken_link\">algorithms from bioinformatics<\/a>,&nbsp;with a recent&nbsp;<a href=\"http:\/\/code.google.com\/p\/automated-protocol-reverse-engineering\/\" target=\"_blank\" rel=\"noopener\">open-source implementation<\/a> combined with statistical techniques;&nbsp;automata algorithms to infer <a href=\"http:\/\/www.iseclab.org\/people\/gilbert\/prospex-oakland.pdf\" target=\"_blank\" rel=\"noopener\" class=\"broken_link\">transitions between states<\/a>;&nbsp;<a href=\"http:\/\/bitblaze.cs.berkeley.edu\/papers\/replayer-ccs2006.pdf\" target=\"_blank\" rel=\"noopener\">static binary analysis<\/a>&nbsp;and <a href=\"http:\/\/www.auto.tuwien.ac.at\/~chris\/research\/doc\/ndss08_protocol.pdf\" target=\"_blank\" rel=\"noopener\">runtime analysis of binaries<\/a>&nbsp;because access to the runtime call stack is very convenient whenever using <a href=\"http:\/\/www.utdallas.edu\/~zxl111930\/file\/AutoFormat_NDSS08.pdf\" target=\"_blank\" rel=\"noopener\" class=\"broken_link\">distributed computing contexts<\/a>. Finally, a very interesting project was <a href=\"http:\/\/research.microsoft.com\/en-us\/um\/people\/wdcui\/papers\/discoverer-security07.pdf\" target=\"_blank\" rel=\"noopener\">Discoverer<\/a>&nbsp;@<a href=\"http:\/\/research.microsoft.com\/en-us\/projects\/reveng\/\" target=\"_blank\" rel=\"noopener\" class=\"broken_link\">Discover@<span class=\"caps\">MSR<\/span><\/a>: they announced very high success rates for very complex protocols (<span class=\"caps\">RPC<\/span> \u2013 <span class=\"caps\">CIFS<\/span>\/<span class=\"caps\">SMB<\/span>), but the tools were never released,<\/p>\n<p style=\"text-align: justify;\"><iframe src=\"\/\/docs.google.com\/viewer?url=http%3A%2F%2Fresearch.microsoft.com%2Fen-us%2Fum%2Fpeople%2Fwdcui%2Fpapers%2Fdiscoverer-security07.pdf&amp;hl=en_US&amp;embedded=true\" class=\"gde-frame\" style=\"width:100%; height:500px; border: none;\" scrolling=\"no\"><\/iframe>\n<\/p><p class=\"gde-text\"><a href=\"http:\/\/research.microsoft.com\/en-us\/um\/people\/wdcui\/papers\/discoverer-security07.pdf\" class=\"gde-link\">Download (<span class=\"caps\">PDF<\/span>,&nbsp;<span class=\"caps\">30KB<\/span>)<\/a><\/p>\n<p style=\"text-align: justify;\">This post would not be complete without the mention of the best inspiration for every reverse engineer in the network field: <a href=\"http:\/\/www.samba.org\" target=\"_blank\" rel=\"noopener\" class=\"broken_link\"><span class=\"caps\">SAMBA<\/span><\/a>,&nbsp;the <em>magnum opus<\/em> of <a href=\"http:\/\/www.samba.org\/~tridge\/\" target=\"_blank\" rel=\"noopener\" class=\"broken_link\">Andrew Tridgell<\/a>, an open-source interoperability suite to let Linux and Windows computers talk together. A book about the protocol and the project, <a href=\"http:\/\/ubiqx.org\/cifs\/\" target=\"_blank\" rel=\"noopener\" class=\"broken_link\">Implementing <span class=\"caps\">CIFS<\/span><\/a>, is as good as any divulgation book can get: he makes it look so easy, even a child could do&nbsp;it.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The black arts of reverse engineering network protocols have been lost. These days, every network protocol seems to be run over <span class=\"caps\">HTTP<\/span> and handling lots of <span class=\"caps\">XML<\/span>: every network engineer of the past decades would just cringe at the thought of&nbsp;it. Complete specifications of network protocols like those offered in RFCs have always been luxuries:&nbsp;[\u2026]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"wp_typography_post_enhancements_disabled":false,"ngg_post_thumbnail":0},"categories":[6,10],"tags":[],"_links":{"self":[{"href":"https:\/\/cerezo.name\/blog\/wp-json\/wp\/v2\/posts\/1253"}],"collection":[{"href":"https:\/\/cerezo.name\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cerezo.name\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cerezo.name\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/cerezo.name\/blog\/wp-json\/wp\/v2\/comments?post=1253"}],"version-history":[{"count":4,"href":"https:\/\/cerezo.name\/blog\/wp-json\/wp\/v2\/posts\/1253\/revisions"}],"predecessor-version":[{"id":1559,"href":"https:\/\/cerezo.name\/blog\/wp-json\/wp\/v2\/posts\/1253\/revisions\/1559"}],"wp:attachment":[{"href":"https:\/\/cerezo.name\/blog\/wp-json\/wp\/v2\/media?parent=1253"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cerezo.name\/blog\/wp-json\/wp\/v2\/categories?post=1253"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cerezo.name\/blog\/wp-json\/wp\/v2\/tags?post=1253"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}