Issued for Abuse: Measuring the Underground Trade in Code ... · $16,150 in revenue for selling...

Issued for Abuse Measuring the Underground Trade in Code Signing Certificates

Kristian KozakFaculty of Informatics

Masaryk Universitykkozakmailmunicz

Bum Jun KwonUniversity of Maryland

bkwonumdedu

Doowon KimUniversity of Marylanddoowoncsumdedu

Tudor DumitrasUniversity of Maryland

tdumitraumiacsumdedu

AbstractmdashRecent measurements of the Windows code signingcertificate ecosystem have highlighted various forms of abusethat allow malware authors to produce malicious code carry-ing valid digital signatures However the underground tradethat allows miscreants to acquire such certificates is not wellunderstood In this paper we take a step toward illuminatingthis trade by investigating the certificate black market fromtwo separate perspectives First we identify 4 leading vendorsof Authenticode certificates we document how they conductbusiness and we estimate their market share Second wedig deeper into the demand for code signing certificates bycollecting a dataset of recently signed malware and by using itto study the relationships among malware developers malwarefamilies and certificates We also establish indirect links be-tween these two data sets by inferring that 5 certificates foundin our signed malware samples had likely been purchasedfrom one of the black market vendors we observed Usingthis approach we document a apparent shift in the methodsthat malware authors employ to obtain valid digital signaturesWhile prior studies have reported that most of the code signingcertificates used by malware had been issued to legitimatedevelopers and later compromised we report that in 2017this method is not prevalent anymore Instead we gatherevidence consistent with a stable underground market thatrepresents the leading source of code signing certificates formalware authors We also find that the need to bypass platformprotections such as Microsoft Defender SmartScreen playsan important role in driving the demand for Authenticodecertificates Together these findings suggest that the trade incertificates issued for abuse represents an emerging segmentof the underground economy

1 Introduction

How can Alice ensure that an executable program de-veloped by Bob has not been tampered with or replaced bymalware As a solution to this problem modern computingplatforms have adopted digital signatures where the trustderives from a Public Key Infrastructure (PKI) This infras-tructure includes trusted third parties known as CertificateAuthorities (CAs) who verify the identity of publishers suchas Bob Bob the software publisher may obtain a code

signing certificate from these CAs through a vetting processwhere his identity is verified After acquiring the certificateBob can sign his software and embed the certificate thatbinds the signing keys to Bobrsquos identity Alice the clientcan then check the validity of the certificate and verify thesignature to ensure that the software really comes from Bob

Although a digital signature does not guarantee that thesoftware is safe to execute it helps to establish trust in theprogram In consequence valid digital signatures can helpmalware bypass platform protections and anti-virus scan-ners [6] [11] A well-known example is the Stuxnet wormwhich was digitally signed using keys stolen from twoTaiwanese semiconductor companies [6] Prior measurementstudies have reported many cases of compromised or abusedcertificates that were embedded in malware [6] [7] [11][20] [23] or potentially unwanted programs (PUPs) [1][14] [16] [28] While publishers of PUPs (eg adware) mayobtain code-signing certificates legitimately from CAs [12][16] [25] malware authors typically aim to conceal theiridentities when signing their samples As this prevents themfrom undergoing the CAsrsquo identity verification process thisraises the question how do malware authors acquire code-signing certificates

A longitudinal study of malware signed between 2003ndash2014 [11] reported that most of the certificates involvedhad been issued to legitimate software publishers and latercompromised as in Stuxnetrsquos case Recent anecdotal evi-dence suggests that code signing certificates are also tradedon underground markets [8] [10] [19] This trade allowsmalware creators to purchase a code signing certificate witha fresh publisher identity and to use it to sign malware

In this paper we present the first in-depth analysis ofthis underground trade considering the whole ecosystemof vendors malware developers and certificate issuers andinvestigating the vendorrsquos market share and the factors thatdrive the demand in the market1 Having inspected 28forums 6 link directory websites 4 general marketplacesand dozens of websites trading black market goods weidentify 4 leading vendors of code signing certificates The

1 A preliminary version of this paper appeared as [15] In a parallelstudy [2] Recorded Future also monitored 4 certificate vendors but did notanalyze in depth the links between these vendors and the signed malwarethat appeared in the wild during the same period

arX

iv1

803

0293

1v3

[cs

CR

] 1

4 Fe

b 20

19

overall activity of these vendors has been increasing in thefirst half of 2017 with one of the vendors setting up hisown e-shop in August 2017 We regularly collected stockinformation and analyzed the sales of this e-shop Duringthe 104 days of our observation period the e-shop obtaineda new certificate every two days on average and collected$16150 in revenue for selling these certificates A new codesigning certificate generally trades for a few hundred dollarsExtended Validation (EV) code signing certificates are alsooffered for a few thousand dollars each

To further investigate the impact of this undergroundtrade on signed malware we collect 14221 signed malwaresamples using the VirusTotal Hunting API [27] and extract1163 certificates from these samples By analyzing thesubset of samples that also carry a trusted timestamp wefind that around 45 of the abused certificates are used tosign malware within the first month after they were issuedThis pattern of abuse is consistent with a black marketthat frequently obtains new certificates and makes themavailable for general consumptionmdashas observed in our stockmeasurements for the code signing e-shopmdashbut would bedifficult to sustain by relying on signing keys stolen fromlegitimate publishers To further corroborate this connectionwe utilize several properties of the certificates to infer that 5certificates found in our signed malware samples had likelybeen purchased from the e-shop during our observationperiod

We analyze the relationships among the actors in thisunderground market by building a graph that connects pub-lisher identities with certificates and signed malware fami-lies We identify a large strongly connected component thatcontains most of the samples and mainly Russian publisheridentities with other components being generally small andwell separated The strong connectivity within the largecomponent suggests a degree of cooperation among thevarious malware developer teams and confidence in theblack market This cluster also exhibits the highest rate ofburning new certificates by using around 60 of them tosign malware within the first month after issuance

The emergence of this underground trade points to agrowth in the demand for code signing certificates from mal-ware authors The requests for certificates on undergroundforums and the marketing messages of the four certificatevendors suggest that a key factor driving this demand isthe need to bypass Microsoft Defender SmartScreen [18] acertificate reputation system built into Windows 10 In turnthis demand has created a market opportunity for specializedvendors who can obtain new certificates regularly and sellthem for immediate use

In summary this paper makes four contributions

bull We conduct an exploratory analysis of the under-ground trade in code signing certificates We illumi-nate its business model marketplaces and pattern oftransactions

bull We report evidence consistent with a shift in howmalware is signed as the demand for new certificatesleads to a growing prevalence of certificates issued

for abuse (rather than compromised certificates asreported in the prior work)

bull We perform a graph-based analysis of the attributesof signed malware and we infer new relationshipsamong malware developers

bull We use indirect evidence to infer that 5 certificatesfrom digitally signed malware found in the wild hadlikely been purchased from the underground market

We utilize our findings to draw lessons about the role ofthe certificate black market in the ecosystem of digitallysigned malware and we discuss concrete proposals forfacilitating the verification of publisher identities We releaseour data set of certificates extracted from signed malwaresamples at httpssignedmalwareorg

2 Problem Statement

21 Code Signing Overview

Microsoft Authenticode [17] is a standard for digitallysigning Windows portable executables (PE) The digitalsignatures allow client platforms to identify the publisherwho developed the software and to ensure the integrityof the signed executables Each signature appended to anexecutable is accompanied by a code-signing certificatewhich binds the signing key to the publisherrsquos real iden-tity These certificates are issued and signed by CertificateAuthorities (CAs) who are trusted to verify the identitiesof the publishers who request code-signing certificates Eachcertificate includes a specification of the dates of issuanceand expiration To avoid the challenge of replacing binarieswith expired certificates in the field software publishersoften submit their binaries to Time-Stamping Authorities(TSAs) and receive unforgeable timestamps proving whenthe binary was signed Signed binaries that include suchtrusted timestamps are considered valid even after the certifi-cate expires Together these roles and procedures representthe Authenticode public key infrastructure (PKI) which isthe basis for establishing trust in Windows executables

Unlike the better studied Web PKI the AuthenticodePKI is opaque as compromised certificates cannot be dis-covered systematically through network scanning and thereis no official list of legitimate software publishers Thisfacilitates abuse allowing miscreants to obtain code sign-ing certificates and to produce valid digital signatures formalicious code Several mechanisms have been introducedto prevent this abuse

Revocation Requirements In 2016 the Certificate Author-ity Security Council (CASC) adopted a set of minimumrequirements for code signing certificates [3] According tothese guidelines a CA must promptly investigate and revokea code signing certificate after being notified that the certifi-cate has been used to sign malware This makes it difficultfor malware authors to reuse code signing certificates

2

EV Certificates In addition to standard code-signing certifi-cates CAs may issue Extended Validation (EV) certificatesTo obtain an EV certificate the publisher has to pass amore stringent vetting process EV certificates convey ahigher degree of trust and are required when signing criticalcode (such as Windows drivers) [24] In consequence EVcertificates are both more valuable and harder to obtain formalware authors

Microsoft Defender SmartScreen To combat malwareMicrosoft Defender SmartScreen [18] assigns reputationscores to executables and Authenticode certificates Whenan executable is downloaded on a Windows 10 machineSmartScreen attempts to assess its reputation before it allowsthe client to launch it The reputation scores take severalfactors into account For example EV certificates receivea good reputation initially Conversely if the applicationthe URL from where it was downloaded or its code signingcertificate appear on a blacklist SmartScreen assigns a badreputation SmartScreen reputations can improve over timefor example when an application garners a track record ofinstallations on multiple hosts without raising suspicions

However if SmartScreen encounters a previously un-known application with a valid digital signature that isvalid but is endorsed by a previously unknown certificateWindows 10 displays a warning dialog to the user beforelaunching the application This represents a challenge formalware developers who aim to avoid suspicion and toremain stealthy Even if they manage to produce a validsignature for their malware they must also ensure that theirAuthenticode certificate has accumulated sufficient reputa-tion in SmartScreen to prevent the user warning

22 Underground Certificate Trade

Despite the anti-abuse mechanisms reviewed above re-cent measurement studies have systematically uncoveredcases of potentially unwanted programs (PUPs) [1] [12][14] and malware [11] carrying valid digital signaturesThese studies have shown that miscreants value the abilityto sign malicious code and that they are able to controla range of Authenticode certificates However this line ofprior work did not shed light on how malware authors obtainthese certificates2

At the same time anecdotal evidence [8] [10] [19]suggests that code signing certificates are traded on under-ground markets However this segment of the undergroundeconomy is not well understood Prior research has docu-mented the emergence of similar underground trades in caseswhere a malicious task required specialized skills and was indemand for example pay-per-install (PPI) services [4] [13][25] allow malware developers to focus on implementingthe malicious functionality and to outsource the malwaredelivery task In contrast the economic forces driving the

2 While publishers of PUPs (eg adware) typically do not conceal theiridentities [12] [16] [25] which may allow them to acquire certificatesdirectly from CAs malware authors cannot rely on this approach

underground trade in Authenticode certificates have not beenanalyzed yet

This is in part due to challenges in collecting data aboutthe certificate black market In particular it is not straight-forward to locate the marketplaces where these certificatesare being traded While prior reports mentioned an onlinemarketplace for anonymous code signing certificates [8]this marketplace was no longer active at the time of thiswriting Christin collected and released a large dataset bycrawling SilkRoad a general marketplace for black marketgoods [5] however our analysis of this dataset did not revealany certificates or other goods related to code signing abuseMoreover we were not able to locate any such goods onother general marketplaces available on the dark web asdescribed later on

23 Research Questions

Our goal in this paper is to conduct an exploratory anal-ysis of the underground trade in code signing certificatesAs a first step toward illuminating this black market weaim to gather evidence that would allow us to formulatehypotheses about the forces driving supply and demand andto suggest further experiments Specifically we ask fourresearch questionsQ1 What is the business model There are three pos-sible business models for monetizing abusive certificates(1) selling the certificates themselves allowing the malwaredevelopers to control them and use them to sign as manymalware samples as they wish (2) providing a signingservice which receives binary submissions from malwaredevelopers and signs these binaries for a fee (in this case themalware developers do not control the certificates) and (3)incorporating code signing into Pay-per-Install (PPI) cam-paigns where the service providers decide which binariesget signed in order to increase their chances of successfulinstallation We aim to determine which model is prominenttodayrsquos underground market and to characterize the supplythe demand and the pricing strategy We are also interestedin what drives the demand for certificatesmdashwhat challengesdrive the malware developers to this black marketmdashas thisreflects the real-world effectiveness of the defenses againstcode-signing abuse (reviewed in Section 21)Q2 What are the relationships among the actors in-volved in the trade We aim to investigate the connectionsamong vendors and customers and to assess the stability ofthe certificate black market Evidence of frequently changingbusiness relationships vendors selling the same certificate tomultiple customers or malware authors hoarding certificatescould indicate that the trust among these actors is low(no honor among thieves) Malware authors cannot dependon such a black market for producing valid signatures re-peatedly In contrast evidence for communities of malwaredevelopers who cooperate and share certificates and whouse newly purchased certificates quickly could point to ahigh degree of confidence in the market Such evidencewould suggest that the current business models are viable

3

and that this black market could play a growing role in thefuture malware landscape At the same time understandingthese underground relationships can suggest interventionsthat would effectively disrupt cyber crime operations [26]Q3 Where do the abusive certificates come from Cer-tificates can be either stolen from legitimate publishers orpurchased directly from the CAs anonymously (eg by set-ting up a shell company or impersonating a legitimate one)Stealing a certificate does not require the malware developerto pay a fee and it may not be difficult to steal from carelesspublishers who do not protect their signing keys but thismethod does not guarantee a reliable supply of certificatesin the future On the other hand obtaining a certificate fromthe CA requires the adversary to pay and should be moredifficult to do for malware developers since they would needto pass the vetting process without revealing their identitiesHowever if adversaries are able to set up a reliable processfor passing the CAsrsquo vetting processes this method wouldlikely scale better with the demand for certificates Priorwork showed that stealing certificates was the prevalentmethod between 2003ndash2014 [11] We aim to collect a newerdata set and to analyze how these trends are shifting toreason about the role of the certificate black market in theproduction of digitally signed malwareQ4 Who controls the abusive certificates As the certifi-cates themselves are the most valuable goods in the systemof code signing abuse we aim to analyze signed malwarein the wild to infer who controls the abusive certificatesmdashwhether it is the malware developers themselves or somethird parties that provide code signing services to malwaredevelopers We also aim to determine if this this evidence isconsistent with the business models observed in the vendoranalysis from Q1

3 Data Collection

For our exploratory research we follow an inductiveapproach by developing and refining our model of the blackmarket based on the data We collect data passively byobserving the certificate black market from multiple vantagepoints We do not interact with any of the actors (egwe do not purchase any certificates or exchange messageswith the vendors) and we do not conduct any experimentsdesigned to influence their behaviormdashwith the exception ofresponsibly disclosing the abusive certificates to the CAs

To answer Q1 and Q2 we start by exploring the blackmarket We inspect black market websites where goodsrelated to code signing abuse are traded analyze vendorsand their business models and observe their activities Toanswer Q2 Q3 and Q4 we analyze samples of signedmalware and their connections to certificates and publishersBecause certificate sales take place in private and vendorsdo not advertise information that would uniquely identifytheir certificates (eg the serial numbers) it is challengingto establish a direct link between the certificates tradedon the black market and the ones used to sign malwareHowever by comparing several distinguishing features of

certificates from the two datasets we infer that a subsetof the certificates we found in malware had likely beenpurchased from one of the vendors we monitor This allowsus to connect better the supply and demand sides of Q2 Inthis section we describe our data collection and curation

31 Black Market Data

To address the data collection challenge discussed inSection 22 we start with a small number of publicly avail-able well-known websites (eg forums and marketplaces)and use these websites to gradually expand our data setFrom these data sets we conduct an investigation on thevendors who trade code signing certificates We then analyzeinformation about goods in stock over time to estimate theportion of the black market that we have observed

Due to the large variety of both structure and anti-crawling protections the black market sites feature an au-tomatic analysis is rather difficult Hence we resort toperforming most of the inspection manually using the sitesbuilt-in search tools (search queries are issued for thefollowing strings of keywords ldquocode signing certificaterdquoldquocode signingrdquo ldquosignrdquo ldquosignaturerdquo and ldquosign certificaterdquowith Russian equivalents of these being search for as wellon Russian speaking forums) Automated data collection isutilized where appropriatendasheg for crawling stock informa-tion to see stock updates over timeForums and Marketplaces We begin with a set of wellknown Internet forums (such as Hackforums) link direc-tory sites (mostly on dark web eg Torlinks) and generalmarketplaces on dark web (such as Dream Market) Wethen discover additional sites by following relevant linkssearching for user or vendor handles and searching for thekeywords related to code signing certificates on Google andother search engines We expand our data set in this fashionuntil we stop discovering new sites or we arrive at closedforums that we are unable to access

In total we inspect 28 forums 6 link directory websitespointing to other more specialized web pages 4 generalmarketplaces dozens of websites dedicated to various blackmarket goods (such as credit cards or PayPal accounts)and one website specialized in selling anonymous codesigning certificates While earlier reports by security com-panies (such as one by InfoArmor [8]) reported on ane-shop with anonymous code signing certificates namedcerts4you this e-shop was no longer accessible duringour research However one of the black market vendorsobserved by us on a forum has set up a new e-shop withanonymous certificates Codesigning Guru during theperiod of our research The sites that we investigated arelisted in Appendix A Screenshots from both forum postsand the e-shop can be found in Appendix BVendors amp Purchases We have observed in total 4 vendorsoperating across many forums (under the same handle) andone e-shop claimed to be set up by one of the observedvendors We gather information about the vendor activity(registration date post date last edit date etc)

4

Specifically we focus on stock updates and vouches Toencourage sales vendors often provide information aboutthe remaining stock which can be aggregated across forumsCustomers on the other hand sometimes provide vouchesndasha claim that they have utilized the services offered bythe vendor and that the offer is not a scamndasha mechanismoften used on the black market where establishing a trustedrelationship is difficult

On the Codesigning Guru e-shop payments forcertificates were handled through the Selly3 a platformthat facilitates purchases of digital products paid by digitalcurrencies Through Selly widget information about thecount of certificates on the stock was provided Hence wehave created a crawler that collected count of certificates onstock every five minutes from 20170825 till 20171207The use of Selly enabled us to crawl the information byaccessing only the Selly widget and not loading the e-shopwebsite itself (in this way our traffic was more difficult tospot as the e-shop owner would have to get the informationabout our periodic visits from Selly itself) This method issimilar to the basket inference proposed in [9] except thatwe infer the vendorrsquos stock of certificates rather than thecontent of the customersrsquo baskets

Information about certificate stock was then used toassess the size of the business and attempt to link theobserved sold certificates with certificates observed in thesecond partndashthe collection of signed malware samples

32 Signed Malware Data

Malware samples were collected over the course ofseveral months Then PUPs and binaries with unverifiablesignatures were filtered out and only the correctly signedmalware was further analyzedData Sources Our dataset was collected using the Huntingfeature on VirusTotal 4ndashin this way reports for all files thatare submitted to VirusTotal when the hunting is runningthat satisfy a given condition are collected In our case thecondition was having more than 10 positive reports and thehunting ran between 20170418 and 20171124 Since thereports from hunting do not provide detailed informationabout the signed binary we re-query VirusTotal with thelist of SHA256 hashes caught by hunting using the privateAPI The private API returns further information aboutthese hashes including the AV detections and the certificateinformationndasheg the issue date the publisher name and thesigning date

Once our filtering (description of which follows) isapplied and we are left with correctly signed malware onlyinformation about certificates are extractedFiltering for Signed Malware To separate PUP frommalware we have used approach introduced in prior work[11] First we have calculated rmal rate as the fraction ofpositive reports from total AVs that were used to scan thesample Then we identified 12 keywords that are often used

3 wwwsellygg4 wwwvirustotalcom

Item Count

Total samples 2117600Malware 188421Correctly signed malware 14221

Samples analyzed 14221Extracted certificates 1163Publisher names 1073Publisher Identities (PIs) 788Non-singleton PIs 114Distinct publisher names in non-singleton PIs 399

TABLE 1 Summary of counts of samples certificates andpublishers as we proceeded through the data preparationpipeline Non-singleton PIs are PIs that correspond to atleast two distinct publisher names extracted from certifi-cates

in AV labels to indicate PUP such as ldquopuprdquo or ldquoadwarerdquoWe have then calculated rpup rate as the fraction of labelsincluding one of these keywords out of all positive AVreports for given sample

While probably not all of the AV labels indicating PUPwere caught with our simple 12-keyword approach Figure2 shows that there is a clear separation between a clusterof samples that appear to be PUP (having higher rpup andlower rmal) and a cluster of samples that appears to bemalware with high confidence (very low rpup an high rmal)Hence we decided to keep the thresholds used in previouswork and recognize samples that have rmal ge 033 andrpup le 01 as malware

Since we want to analyze malware where the develop-ers successfully managed to abuse code signing we thenperform further filtering and keep only samples where thesignature was successfully verified by VirusTotal (that isthe ldquoVerifyrdquo flag says ldquoSignedrdquo in the sigcheck field) Table1 provides the summary of the malware samplesResponsible Disclosure We have reported all the certifi-cates we extracted from signed malware to the issuing CAs

4 The Code Signing Black Market

As explained in Section 31 the black market appears tobe dominated by 4 vendors (AndashD) whose presence rangesfrom from 2 to 10 forums We start our analysis by inves-tigating the business practices of these vendors aiming toanswer our first research question Table 2 summarizes theperiods of activity of these vendors the apparent sizes oftheir businesses and the products they offer

41 Vendors

Presence Vendor D operated on English speaking forumsonly (2 forums in total) vendors A C operated on Russianforums only (10 resp 3 forums) and vendor B the mostactive one operated on both English and Russian forums(5 English and 3 Russian) though he indicated to be ofRussian origin at one point mentioning he was ldquoexpandinghis business [to the English forums]rdquo

5

Figure 1 Simplified diagram of the signed malware analysis pipeline Samples were collected on VirusTotal (1) then PUPswere filtered out (2) and only malware with a valid signature was kept (3) Samples were then labeled using AVClass andclustered using Publisher Identities (4) Finally Malware Map was built (5) to analyze the relationship between the samplescertificates and Publisher Identities

Vendor Activity amp Presence Inferred Sales Volume Products

First Joined Last Reply Forums Vouches Updates E-Shop Item pieces Price ($)

A 2011-09-07 2015-05-25 3 0 - -Standard 1 1000Standard 10+ 800Standard 15+ 700

B(Codesigning Guru) 2016-01-08 2017-08-08 8 3 7+ 41

Standard (Comodo) 350Standard (Thawte) 500EV 2500

C 2016-09-09 2017-01-27 10 0 - - EV (earlier posts) 1600EV (current) 3000

D 2017-03-07 2017-08-13 2 4 7+ -

Standard 1 400Standard 5 1700Standard 10 3500w SmartScreeen rep 1 800w SmartScreeen rep 5 3700w SmartScreeen rep 10 7000

TABLE 2 The leading black market vendors We collected the forum activity on 20170814 and the e-shop sales from20170825 till 20171207 Because all the sales take place in private vendors that appear inactive on the forums may stillbe in business and we cannot observe the sales volume directly We infer the sales volume from vouches updates of countsof certificates on stock and e-shop stock information crawl (where applicable)

Generally our research started on the more known andpublicly accessible forums However we have encounteredmost of the vendors already there and as we continued toprogress to forums that are more difficult to access we stillencountered the same vendors again and again (Eventuallywe have reached also forums that we were not able to accessdue to the closed community) In general vendors usuallystart on the more protected forums and then reach out to themore public ones in order to expand their business

During our observation period the vendor B has broad-ened his presence by setting up a new e-shop with anony-mous certificates on Codesigning Guru at the begin-ning of August 2017 Goods and prices generally matchedwhat these vendors were selling on the forums The vendoroften promoted the URL of the e-shop on the forums todirect potential customers there

Activity Generally the activity of the vendors was surging

While the oldest vendor A appeared not to be very activeanymore with all posts from 2015 with no updates the twovendors with the largest presence B and D were buildingit in the last months The three more recent vendors B CD all had half or more of their posts (threads) establishedafter May 2017

The two most active vendors B and D regularly updatedtheir posts (approx once or twice per month) and providedreplies and stock updates On major forums such a threadoften contained over 10 public comments of forum usersand over 10 replies and updates from the vendor

Mechanisms Vendors generally start a new thread on aforum detail their offerings in the first post and then updatethe post over time with new features price changes etcOther users ask questions and provide vouches (reviews ofsuccessful purchases) Vendors answer questions and some-times provide updates regarding their stock All purchases

6

Figure 2 Graph showing separation between malware andPUP clusters White color represents the lowest concentra-tion of samples violet the largest Black rectangle shows thethreshold values and marks the area of samples that wereconsidered malware

take place in private usually over jabber or messagingservice built into the forum platform (which often utilizesjabber as well) Hence estimating sales and dates of lastactivity is difficult

Vendors generally use the same handle (username)across the forums to keep their reputation The format andcontent of the post are often exactly the same For linkingvendor identity across forums we have used vendor contactinformation handle post content and post formatndashcontactinformation were sometimes missing (relying on the forumrsquosbuilt-in messaging system) then we relied on the otherpropertiesBusiness Model At all vendors the business model wassimple selling anonymous code signing certificates This issurprising as previous reports [8] had reported Signature-as-a-Service providers in the past However even thoughwe searched for general keywords such as ldquosignaturerdquo andldquocertificaterdquo (as described in Section 31) we were not ableto find evidence of any other business model aside fromselling certificates

All vendors claim that their certificates are freshly is-sued that is that they have not been stolen in any way butobtained directly from a CA Further all vendors claimedthat they sell one certificate into the hands of just one clientand some even offered free or cheap one-time reissue if thecertificate was blacklisted too soon Vendors did not appearto be concerned with revocation often stating that it usuallyldquotakes agesrdquo until a CA revokes an abused certificate

42 Goods and Deals

Goods Inventories of all four observed vendors are rathersimilar Vendors A D focus on general code signing cer-tificates vendor C offers EV certificates only and the most

active vendor B (who is behind the Codesigning Gurue-shop) offers both types of certificates

All vendors claim that certificates are fresh obtaineddirectly from a CA For customers this should be easy toconfirm by checking the issue date as certificates appearedto be sold very soon after being put on the stock by thevendor Some vendors even claim to obtain the certificate ondemand having the certificate issued once a customer payshalf of the price Interestingly vendor A even claims thathe always has a few publisher identities prepared and thecustomer can then choose which of these publisher nameshe wants to have his certificate issued onCertificates with Built SmartScreen Reputation Espe-cially on forums that appear more beginner centered lessproficient users often appeared to not have a clear ideaabout how code signing works but rather just seeing thatSmartScreen blocked their unsigned malware from beinglaunched or produced undesirable warnings Hence on theseforums vendors often explain how code signing works intheir posts and include keywords like ldquobypass SmartScreenrdquoin their post titles indicating that this is what less proficientmalware developers search for

With general (non-EV) code signing certificatesSmartScreen reputation first needs to be built-up before mal-ware can be installed and executed without issuing warningsVendor D offers certificates with an already built reputationfor double the price of a certificate without reputationVendor B (Codesigning Guru) does not offer certifi-cates with positive reputation unless a special demand isfiled Moreover FAQ on Codesigning Guru mentionsthat approx 2000-3000 installs of benign files on Win-dows 10 systems are needed in order to establish sufficientSmartScreen reputation to avoid warningsPrices The prices for standard code signing certificatesrange from $350 to $500 Vendor D offers certificates fromvarious CAs (citing Comodo Thawte DigiCert Symantec)all for $400 The most active vendor B differentiates pricingby CA a Comodo certificate costs $350 while Thawtecertificates (which are more trusted according to the vendor)cost $500 each Appendix C includes a sample post fromthis vendor The vendors often offer bulk discounts

EV certificates are considerably more expensive Earlierposts by vendor C list a price of $1600 while more recentposts by the same vendor offer EV certificates for $3000Vendor B sells EV certificates for $2500 (both in the forumsand in the Codesigning Guru e-shop) The EV certifi-cates come pre-installed on USB tokens which the vendorsthen send to the buyers by post As CAs require morerigorous vetting procedures before issuing EV certificatesthe supply is more limited than for standard certificates Inparticular vendors A and D do not offer EV certificatesThe posts advertising EV certificates that we collected fromvendors B and C did not specify the issuing CA

43 Sales Volume

Forums Vouches amp Stock Updates As all purchases onforums take place in private it is difficult to estimate sales

7

Certificates Total Average per month

Items sold Revenue ($) Items sold Revenue ($)

Comodo 29 10150 836 2928Thawte 12 6000 346 1731Total 41 16150 1183 4659

TABLE 3 Sales volume recorded on Codesigning Gurubetween 20170825ndash20171207

counts precisely In Table 2 we report the number of vouchesfor each vendor as vouches are the method for trust on theblack market with users claiming they have used the serviceand providing a short review of it However the customersmay choose not to submit a vouch even after successfultransactions Since we have observed very few vouches (2for vendor B and 4 for vendor D) estimating sales preciselywith this method is not feasible

A more fruitful resource are stock updates that are some-times provided by the vendors (even though they should betaken with a grain of salt since they obviously might not betrue) For the most active vendor B we have aggregated intotal 17 stock updates across 6 forums We have for exampleobserved that at one point four code signing certificates weresold in just four days and in another period no certificatesold in 14 days However while stock updates might begood to get a very general impression about sales and pro-vide information over short-term periods estimating long-term sales is challengingE-Shop Stock Information Crawl Our third approachis to analyse the stock information on the CodesigningGuru e-shop By using the method described in Section 31we can infer when certificate sales are completed on this site

Backed by vendor B the most active of the four blackmarket vendors the e-shop offers Comodo and Thawtestandard code signing certificates as well as EV certificatesissued by unspecified CAs While EV certificates are ob-tained on demand the e-shop publishes stock availabilityfor standard certificates During our 104-day observationperiod (from 20170825 to 20171207) we have recorded41 standard certificate sales The average rate of sales forthe e-shop was 118 certificates per month bringing in atotal revenue of $16150 (see Table 3)

The demand appears to be rather high as the stockavailability suggests that certificates are often sold withinone day of being put on the stock Additionally as thevendor claims that the certificates are freshly issued we caninfer the issue dates of the certificates sold on the e-shopwe analyze this information in Section 6

5 Signed Malware Analysis

While conducting the study on the code signing blackmarket we observed that the market is active and thevendors have a potential of scaling the business This raisesthree questions about the demand for certificates (1) howconnected the whole ecosystem ismdashwhether an abused cer-tificate could somehow lead us to another abused one and

if there are indications of cooperation among malware de-velopers (2) how prevalent the certificates issued for abuseare among the signed malware and (3) who takes control ofthose certificates

We focus on malware since it benefits the most fromusing anonymous certificates as code signing helps malwarebypass protections [11] and anonymous certificates do notreveal the identity of the malware creators We collectmalware with valid digital signatures as described in sec-tion 32 To analyze the signed malware ecosystem we intro-duce an abstraction the Signed Malware Map that capturesthe relationships and communities among the malware andthe abusive certificates We then map the characteristics andconnections in the system in Sections 51 and 52 examineprevalence of certificates issued for abuse in Section 53and investigate who controls the certificates in Section 54

51 The Signed Malware Map

Here we introduce the Signed Malware Map which cap-tures the code signing abuse ecosystem The signed malwaremap is a graph reflecting the relationship among malwarefamilies and the certificates they use We first introduce thepreparation of the entities that the graph consists of Thenwe describe the construction of the signed malware mapClustering Publishers We observe that multiple certificatesfrom signed malware tend to be issued to what appearsto be one publisher company identity with slight variationin the publisher name A similar finding was reported forPUP publishers [14] For the malicious actor who requeststhese certificates from CAs this provides the benefit of nothaving to set up multiple fake identities while preventingthe revocation of multiple certificates belonging to the samemalicious publisher

We tried to apply the publisher clustering techniqueproposed in prior work [14] which utilizes the normalizededit distance However we found out that it was not aneffective approach in our case because (1) publishers withdifferent company names with low edit distance (for ex-ample companies with short names) ended up in the samecluster even though the publisher identity seemed to bedifferent and (2) most of the certificates did not containprecise street address to help with the clustering

Hence we have developed our own approach Most ofthe variations in publisher names appeared to be in non-lettercharacters such as commas dashes quotes and sometimeseven backticks More interestingly the exact same publishernames have often appeared with different suffices to indicatethe company type such as rdquoLtdrdquo and rdquoOOOrdquo (Russianversion of Ltd)ndasheven though publisher country stays thesame An example might be the following three publishernames

bull Ltd rdquoVet Faktorrdquobull OOO Vet - Faktorbull LLC rdquoVET FAKTORrdquo

Our technique for clustering the publishers then comprisedof (1) removing all company type identification substrings

8

such as rdquoLLCrdquo and rdquoCordquo (2) removing all non-letter char-acters such as dashes commas and spaces (3) converting thestring to lowercase and (4) comparing the resulting stringsfor exact match We then examined manually a subset of thesamples to validate the publisher-name clusters producedby our technique We refer to a cluster of publisher names(matched as described above) as a Publisher Identity (PI)

Over a third (3719) of publisher names extracted fromcertificates belong to a non-singleton PI (that is there existsa different publisher name that matches the same PublisherIdentity) Count of unique publisher names PIs and otherstatistics are listed in Table 1

AVClass Labeling Only clustering of the publishers byitself does not provide enough connections between thecertificates and publisher names Hence our next step tobuild analyzable communities was to label the samples withmalware family names they are recognized as belonging tondashwe used AVClass tool [21] to extract the malware familyinformation from AV labels This step enabled us to build thecomplete map (network) of malware description of whichfollows

For the labeling process AVClass was left in its defaultconfiguration apart from whitelisting the word rdquoconfidencerdquoso that AVClass would not consider it a malware familyname This was determined by analysis of the labeled datandashafter performing the labeling we have examined outlierfamilies Family named rdquoconfidencerdquo had most samples andcertificates but when we inspected the AV labels that led toindividual samples being labeled as rdquoconfidencerdquo family wediscovered that this was due to many AVs providing confi-dence level in the label (eg rdquomalicious confidence100rdquo)and AVClass accidentally considering this a family nameHence we whitelisted the term rdquoconfidencerdquo so that it wouldnot be treated as a malware family name (Other outlierswith many samples and certificates appeared to representgenuine large malware campaigns such as the labeled Zusyspyware Loskad trojan and iCloader and Loadmoney beingon the border of PUP and droppers)

Building the Signed Malware Map To analyze the rela-tionships and communities of PIs and certificates we havebuilt a graph There are two types of nodes (1) nodesrepresenting a malware family and (2) nodes representinga PI There is an edge between a malware family and a PIif there exists a sample that belongs to this malware familyand is signed by a certificate issued to this PI Moreoverthere is an edge between two malware families if there aretwo samples each of them belonging to one of the familiesthat are signed with the same certificate (Hence the graphis not bipartite) To further simplify the graph we excludeall singleton malware familiesndashthat is families that containjust one sample as they do not provide any new connectionsbetween publishers and malware families and hence do notcontribute significantly to the overview of relationships

The assumption behind this construction is that if twosamples of malware belong to the same malware familythere is a high chance that the same team of malwaredevelopers is behind both of the samples

Figure 3 A smaller component consisting of 3 malwarefamilies (blue) and 5 PIs (yellow)

We create direct connections between malware familiesthat share a certificate to avoid introducing a third typeof nodes representing certificates which would add furthercomplexity to the graph The Signed Malware Map reflectsthe fact that a connection through a certificate is strongerthan through a PI In the first case the same entity con-trolling the certificate signed malware from both families(whether itrsquos the developer himself or some 3rd party) Inthe second case a 3rd party might have just obtained twodifferent certificates for the same PI and shared it with twodifferent malware developer teams (without actually seeingwhat they are signing with it)

52 Analysis of the Code signing Abuse Ecosystem

First we analyze the structure of the Signed MalwareMap by inspecting the strongly connected componentsSuch components may arise from stable business connec-tions among the participants in the ecosystem We haveobserved one dominant component that contains most ofthe samples as well as certificates (issued mostly to Rus-sian publishers) The remaining nodes are parts of a largenumber of small components that always contain only asmall number of samples and certificates (issued mostly toChinese publishers) While we expected to observe similardistributions distributions of characteristics among malwaresamples regardless of which component they are part ofwe find interesting differences The distributions of somecharacteristics are completely different (eg publisher coun-try issuer) and others are more similar but still featuresignificant differences (eg issue-to-abuse interval) Becausethese contrasts may reflect differences in the use of code-signing certificates and in the confidence in the certificateblack market we divide the nodes into these two groups inthe rest of this sectionComponents Interestingly the map consists of one largecomponent (we will refer to it as Major Component orMC) which encompasses 6973 certificates and 6053PIs Other than that there are 196 small components noneof which contains more than 11 certificates and 9 PIs Atypical smaller component consisting of 3 malware familiesand 5 PIs is shown in Figure 3Connectivity of MC Interestingly if we remove the PInodes and consider the induced subgraphmdashthat is weconsider only nodes representing malware families andedges between them that represent certificates shared amongfamiliesmdashmost of the MC remains strongly connected with

9

Item Total in MC in MC

Samples 14221 13337 9378Certificates 1163 811 6973PIs 788 477 6053Families 317 157 4953

TABLE 4 Summary of counts of samples certificates andpublishers in the major component Only non-singleton fam-ilies (that is families with more than one sample) are takeninto account MC stands for Major Component

Figure 4 Distribution of publisher countries based on com-ponent

only 7 malware families out of total 157 getting separated(Unfortunately the graph is too big to be included in thispaper)

This strong connectivity suggests that malware developerteams in MC either share certificates or use the same 3rdparty service for signing their files As discussed in Section4 we did not find evidence of signing services and thevendors claim that the certificates are freshly issued (a claimwe further investigate in Section 53) In consequence it ap-pears that malware developer teams themselves control theircertificates Then this type of strong connectivity wouldindicate strong cooperation among malware developer teamsbehind samples from the MCCharacteristics Table 4 summarizes the distribution ofsamples certificates etc among the Major Component andother smaller components As we can see a majority ofsamples as well as abused certificates are in the MC At thesame time as shown in Figure 4 the majority of certificatesin MC are issued to Russian and Ukrainian publisherswhile in the smaller components Chinese certificates aredominant Overall Russian publishers are by far the mostprevalent in abused certificates This corresponds to theobservation that 3 of the 4 black market vendors targetRussian speaking audiences (see Section 41)

Figure 5 shows the distribution of abused certificatesacross CAs Most of the certificates from our corpus ofsigned malware had been issued by Comodo AdditionallyComodo and GoDaddy have more certificates in MC while

Figure 5 Distribution of CAs based on component

other CAs have the majority of their abused certificates inthe smaller components

53 The Prevalence of Certificates Issued for Abuse

We aim to estimate which method of abusemdashlegitimatebut compromised certificates or certificates issued straightfor the use of malware authorsmdashaccounts for most of thedigitally signed malware from 2017 Prior work [11] iden-tified compromised certificates by searching for certificatesused to sign both benign files and malware More recentlySmartScreen protections have been built into Windows 10at operating system level [18] driving vendors and mal-ware developers to build up SmartScreen reputation byfirst signing and distributing benign files before using thecertificate to sign malware (see Section 4) In consequencethe existence of both benign files and malware signed withthe same certificate is no longer a strong indication that thecertificate was compromised

We therefore introduce a new technique for estimatinghow many certificates are issued straight to abusers Wemeasure the delay between certificate issue date and the datewhen the certificate was first abused Often malware devel-opers utilize Trusted Time Stamping to extend the validity oftheir malware past the certificatersquos expiration date In total603 out of the observed 1163 abused certificates have beenused to sign at least one time-stamped file We utilize thesecertificates to compute an upper bound for the issue-to-abuseinterval as neither the trusted timestamps nor the issue datescan be tampered with Since the Time Stamp is signed bya Time Stamping Authority it guarantees that the piece ofmalware was signed at this date For each certificate weselect the file that has the earliest date of Time Stampingand use this date as the date of abuse

Hence we know that certificate was either issued straightto abusers or compromised at some point during this inter-val For stolen certificates we expect that they are uniformlylikely to be stolen and abused at any point during theirlifetime as suggested by Figure 5 from [11] Howevercertificates issued straight to abusers are more likely to be

10

Figure 6 Interval from issue date to the date of first malware being signed and timestamped with a certificate

sold and used soon after being issued In consequence aspike of abuse during the first couple of months wouldbe inconsistent with a prevalence of compromised certifi-cates instead these certificates would more likely have beentraded on the black marketResult Figure 6 shows the distribution of the time-to-abuse(in months) computed as described above and groupedaccording to whether the certificate is in MC or not Bothgroups show a tendency for abusing certificates within thefirst two months after issuance with more pronouncedspikes for the MC

Code signing certificates are usually issued for 1 2 or3 years The distribution of the time-to-abuse reflects thisfact Between 3ndash12 months and between 13ndash24 monthsthe time-to-abuse is close to a uniform distribution (this isalso true during the 25ndash36 month interval excluded fromthe figure) However there is a sharp spike in the 1st and2nd monthmdashindicating that a large number of certificates areabused quickly after they are issued Overall from amongall the certificates we extracted from timestamped malwarearound 55 are abused in the first two months (with around45 in the first month) While there are significant spikesfor certificates outside of MC certificates in MC exhibit thesharpest drop after the second month in MC more than 10times more certificates are abused in the first month thanon average in months 3-12 Figure 7 shows the same phe-nomenon as a cumulative distribution among all certificatesthat are eventually used to sign malware in MC around 60are abused in the first month and around 70 in the firsttwo months after issuance

This result corroborates the vendorsrsquo claims that theysell fresh certificates discussed in Section 4 Moreover thistiming pattern is unlikely to arise if most certificates usedto sign malware are legitimate but compromised certifi-cates The certificates abused within the first month wereprobably obtained with the aim of signing malicious codeAdditionally the early spikes in the distribution representlower bounds for the number of certificates issued straight

to abusers as some malware authors may be saving thesecertificates for later use

Outside of MC more certificates are abused duringmonths 3ndash12 than inside MC This could correspond toa higher reliance on compromised certificates but it couldalso reflect a greater reluctance for burning the certificatesquickly as signing malware may result in the certificatebeing revoked or losing SmartScreen reputation In contrastthe prevalence of certificates abused within 1ndash2 months inMC suggests that the malware authors from that communityare not worried about burning their code signing certificatesThis may reflect a confidence in a reliable supply of freshanonymous certificates perhaps due to tight business con-nections among underground actors contributing to a higherdegree of mutual trust

In consequence the distribution of the time-to-abuse isconsistent with the hypothesis that most of the certificatesfound in our corpus of signed and timestamped malwarehad been issued straight to the black marketmdashat least fora tight-knit underground community that accounts for mostof the digitally signed malware

54 Certificate-Controlling Parties

Our next goal is to infer who controls certificatesmdashwhether malware developers themselves control them orif they utilize third-party signing services While we didnot find any references for signing services during ourinvestigation of the black market from Section 4 we aim tocorroborate this observation with evidence from the signedmalware analysis Specifically if one certificate signs sam-ples from a large number of families this would point to athird party who controls the certificates and offers signingservices to multiple malware developers On the other handif a certificate is used to sign only malware from only asmall number of families then would be consistent witha development team that is responsible for these malwarefamilies and that controls the certificate

11

Figure 7 Percentage of certificates abused within specifiednumber of days after being issued

Certificate Point of View Figure 8 shows that most of thecertificates are used to sign fewer than 40 samples among4 or fewer familiesmdashan indication that they are probablyin the hands of the specific development team behind thesefew families

Although there are some outliers that signed malwarefrom more than 10 families (with 2 of the certificates havingmore than one hundred) it seems that signing lot of malwareacross many families is not the trend We performed amanual inspection to see what families are signed with theseoutliers and the families are mostly singletons (families withonly one sample) This fact points to two possibilities eitherthese samples were not recognized by AVs as belongingto one specific familydevelopment team even though theyactually do belong to one team or some signature providingservices are used for malware campaigns that comprise ofonly one or few samples However most of the certificatesare used to sign malware across few families which is con-sistent with a business model where malware developmentteams usually acquire code signing certificates for their ownuse perhaps by turning to the black market vendorsMalware Family Point of View Observing the situationfrom the other side Figure 9 provides insight on the problemfrom point of view of malware families Similarly mostfamilies use fewer than 10 samples and fewer than 5 certifi-cates suggesting that there is usually not much variation incertificates used to sign malware from one family This isalso consistent with the hypothesis that each developmentteam controls its own certificates and does not have accessto a large inventory of code-signing certificates Howeverwe also observe some outliersmdashmalware families with over500 samples and 100 certificates in some cases Figure 9labels the families that represent the biggest outliers

6 Linking Abused and Traded Certificates

To gather additional evidence for the role of the cer-tificate black market in the landscape of digitally signed

Figure 8 Samples and count of families per certificate

Figure 9 Samples and count of certificates per malwarefamily

malware we aim to determine if some of the certificatesfrom our VT dataset had been purchased from the vendorswe monitored Unfortunately it is challenging to identifyprecisely which certificates originated from the black mar-ket Establishing a direct link between a traded certificateand a certificate extracted from a malware sample wouldrequire matching a unique identifier for the certificate (suchas the serial number) but the vendors do not provide suchinformation Instead we establish indirect links by matchingseveral certificate properties which would be unlikely tocoincide by chance Specifically we search our corpus ofsigned malware samples for certificates that probably hadbeen purchased from the Codesigning Guru e-shopData Overview As explained in Section 31 we collectedthe stock information on the black-market certificate e-shop at 5-minute intervals We made the observation thatindividual certificates were always added to the stock one-by-one Even when 4 certificates were added within 4 hours

12

in one morning each of them was put on stock separatelywith gaps of at least 10 minutes between the stock updates

According to the e-shop owner (vendor B) these certifi-cates are fresh acquired directly from CAs We conjecturethat the owner adds certificates to the stock right afterobtaining them with the time required for completing theissuance procedures accounting for the delays between stockupdates This would imply that the date when a certificatecomes on stock usually corresponds to the certificatersquos issuedate

Besides the likely issue date two additional pieces ofinformation can help us narrow down the list of certificatesthat could have been traded on the e-shop For each certifi-cate offered on the e-shop we also know the issuing CAas the stock availability is provided separately for Comodoand Thawte certificates (see Table 3) Furthermore the e-shop owner claimed on a forum that all certificates sold onCodesigning Guru were issued to British publishersLinking the Certificates We compare the dates whenindividual certificates came on stock to the issue dates ofcertificates in our VT dataset of signed malware Amongall the abused certificates matching the criteria describedabove we compute the fraction for which the issue datescoincide with e-shop stock updates This limits the analysisto certificates that are used to sign malware shortly afterthey are issued this usage is not uncommon as discussedin Section 53

We find a strong correlation for Thawte certificates Dur-ing our observation period (from 20170825 to 20171207)11 Thawte certificates came on stock in the e-shop How-ever 2 of these certificates were added during the same5-min interval and 2 other certificates were added duringthe same day but at different times In other words Thawtestock updates occurred on only 9 days which represent 87of the days within our observation period During the same104-day period 145 certificates from our VT dataset wereissued Out of these 10 (69) are from Thawte and 11(76) have a British publisher 5 certificates used to signmalware from the VT dataset meet all our matching criteriathey were issued by Thawte to a British publisher during ourobservation period If the CA is equally likely to issue acertificate on any day during this period the likelihood thatthe certificatersquos issue date coincides with an e-shop stockupdate by pure chance is 87 In fact each of 5 matchingcertificates was issued on a date when a Thawte certificatewas added to the e-shoprsquos stock the likelihood to observethis by chance is 00005Inspecting the Publishers As the abused certificates thathad likely been purchased from Codesigning Guru representa substantial portion of the Thawte stock sold there (5 outof 12 certificates or 42) we take a closer look at thepublishers named in these certificates to gain additionalinsights into vendor Brsquos methods We search these pub-lishers in the beta version of the British public register ofcompanies5 All the publisher names correspond to youngcompanies some incorporated around a month before their

5 betacompanieshousegovuk

code signing certificate was issued We did not find evidencesuggesting that these companies have software developmentas their primary focus However we were not able tofind out the companiesrsquo contact information so we couldnot confirm their need for using code signing certificatesWe hypothesize that these publishers correspond to either(1) shell companies incorporated by a malicious adversarywho uses them to request code signing certificates in aneffort to conceal the ownerrsquos real identity or (2) legitimatecompanies that the adversary is able to impersonate perhapsby using data mined from the public register

7 Discussion

Our results suggest that it is possible for specializedvendors to set up a reliable process for obtaining code-signing certificates from CAs The underground trade isgrowing and at least one segment of the customer baserepresented by the MC cluster demonstrates a degree ofconfidence in the reliability of the certificate supply Whilethis confidence may be the result of tight business connec-tions among the actors in this cluster in the future othermalware authors may turn to this black market as longas the vendors continue to provide a reliable supply ofcode signing certificates This warrants further investigationinto the methods the vendors use to pass the CAsrsquo identityverification processes and into the best ways to prevent theabuse Based on our exploratory analysis we suggest twoways to raise the bar for underground certificate vendorsRevoking Matching Publisher Names Our research showsthat for over a third (3719) of publisher names wehave extracted from the abused certificates more abusedcertificates exist that use in general the same publishername with only slight modifications (eg added backticksour quotes around the publisher name) Hence if a CAfinds out a certificate has been issued straight to the blackmarket and should be revoked it would be desirable to alsoinvestigate other certificates issued for similarly-named PIsand to revoke them if necessaryStandardization of Publisher Name Format Since com-paring publisher names as we did in our research is not veryefficient for large datasets it would be desirable to standard-ize the way the publisher name is listed in the certificate(withwithout commas quotes in the same way as it is listedin a public registry etc) to make comparing publishers andrevoking all certificates issued on (for example) one boguscompany identity easier

8 Related work

We discuss related work in the three key areas (1)measuring anonymous online marketplaces (ie black mar-kets) (2) code signing abuse and (3) analysis of economicrelationships using graphsBlack market studies Christin [5] measured Silk Roadone of the largest anonymous online black markets in 2011and 2012 the black market was operated as a Tor hidden

13

service He found that narcotics were mostly and activelysold and traded in the black market and that the revenue ofthis black market was $12 million a month He also showedthat the daily sales and the number of the items tradedon the black market were increasing during his observationperiod In October 2013 Silk Road was terminated thenother anonymous online marketplaces (eg Silk Road 2Sheep Marketplace etc) started appearing and taking overthe anonymous online market service of Silk Road Sosca etal [22] analyzed how the Silk Road shut-down affected theanonymous online marketplaces by measuring 16 alternativeanonymous black markets Their observation is in line withChristinrsquos study most trading items (70) were narcoticsThey also found that only a few vendors were very success-ful while most vendors earned less than $10000 in theirentire observation period

While lots of physical merchandise (eg the aforemen-tioned narcotics) are sold on these general black marketssuch as Silk Road we were not able to find any offer forcode signing certificates on such marketplacesndashthese appearto be sold usually either on forums or dedicated e-shops

Code signing PKI abuse There are a few attempts toexamine the code signing PKI abuse and factors FirstSophos [28] measured the signed Windows PE files theycollected between 2008 and 2010 They found that thenumber of signed malicious PE files (eg Potentially Un-wanted Programs malware etc) was increasing in theirstudy interval In line with Sophosrsquo study Kotzias et al [14]and Alrawi et al [1] also observed that most of the signedmalicious PE files they examined were PUPs While theabuse factors were not discussed in the previous studiesKim et al [11] focused solely on signed malware (excludingPUPs) and the abuse factors They showed that most signedmalware resulted from stolen private keys and also that alot of signed malware contained an invalid or malformedsignature

Compared to these studies we focused only on malwarethat is correctly signed and attempted to document theblack market where the certificates came from which wasnot previously studied While a majority of samples in thedataset used by Kotzias et al were PUPs we excluded PUPsaltogether and focused only on malicious signed binariesCompared to Kim et al we focused solely on malwarewhere the signatures can be properly verified and used anewer dataset that enabled us to amass much more samplesof signed malware and document the current trendsndashwhileKim et al have shown that in 2014 and earlier most certifi-cates used to sign malware were obtained by stealing privatekeys we document a new trend of obtaining anonymouscertificates straight from the CAs that appears to have gainedtraction during 2016 and 2017

In a separate study conducted in parallel with oursRecorded Future has analyzed the underground markets trad-ing code signing certificates [2] While their work focuseson monitoring the certificate vendors and does not collector analyze a dataset of signed malware their findings are inline with oursmdashthere are 4 certificate vendors who obtain

their certificates directly from the CAs instead of stealingexisting certificates from software developers The RecordedFuture study also reports an interaction with a vendor tosign a new malware sample with a recently issued Comodocertificate in order to confirm the effectiveness of digitalsignatures in bypassing anti-virus detections In contrast wecollect data passively to avoid influencing the behavior ofthe underground markets

Graph analysis Kotzias et al continued their study ofPUPs in 2016 and 2017 by analyzing a network of peopleand companies behind Spain-based PUP campaigns usingentity graphs and leveraging also information available incode signing certificates used to sign the PUPs [13] Bycombining this information with company registers andother sources they were able to comprehensibly map thePUP network

The core difference to our work here is that PUP distrib-utors may sign their software using their own identity (anda legitimately obtained certificate) as PUP programs likeadware are not obviously malicious Malware developerson the other hand are keen to conceal their identity hencethe need for the underground markets with anonymous codesigning certificates that we were trying to provide an in-sight in Therefore in our analysis we used the certificatesand publisher information in them to create links betweenmalware families the developer teams behind them and theblack market vendors But we did not attribute these cam-paigns to real-world people and companies as the identityof publishers might have been either forged or stolen

9 Conclusions

We conduct an exploratory analysis of the black marketstrading code signing certificates We investigate 4 blackmarket vendors with one of them setting up an e-shopspecialized on code signing certificates and selling morethan 10 certificates per month with the total of $16150in revenue during our observation period Using a datasetof signed malware that we have collected we support thevendorsrsquo claims that the certificates are obtained straightfrom the CAs by showing that around 45 of all abusedcertificates are used to sign malware within a month afterthey are issued We further analyze the relationships betweenthe certificates publishers and malware families to show thatindividual developer teams appear to be in control of theirown certificates

Our research provides evidence consistent with a shiftin the code-signing abuse ecosystem toward obtaining thecertificates straight from the CAs Once specialized vendorsestablish reliable processes for obtaining certificates in thismanner this model scales better with the demand than amodel that relies on compromised certificates We suggesttwo practical ways to make this abuse more difficult search-ing for certificates issued to similarly named publishers andrevoking them as appropriate and standardizing the formatfor publisher names More importantly our exploratory re-sults explain the existence of a growing black market for

14

code-signing certificates and they warrant further investiga-tion into the methods the underground vendors use to obtainfresh certificates

Acknowledgment

The authors would like to thank the anonymous review-ers for their constructive suggestions Nicolas Christin forsearching the SilkRoad data set for goods related to codesigning abuse and VirusTotal for allowing us to collectour corpus of signed malware samples This research waspartially supported by the National Science Foundationthrough award CNS-1564143 and two travel grants

References

[1] O Alrawi and A Mohaisen Chains of distrust Towards under-standing certificates used for signing malicious applications InProceedings of the 25th International Conference Companion onWorld Wide Web WWW rsquo16 Companion pages 451ndash456 Republicand Canton of Geneva Switzerland 2016 International World WideWeb Conferences Steering Committee

[2] A Barysevich The use of counterfeit code signing certificates is onthe rise httpswwwrecordedfuturecomcode-signing-certificates2017

[3] CA Security Council Leading certificate authori-ties and microsoft introduce new standards to protectconsumers online httpscasecurityorg20161208leading-certificate-authorities-and-microsoft-introduce-new-standards-to-protect-consumers-online2016

[4] J Caballero C Grier C Kreibich and V Paxson Measuring pay-per-install The commoditization of malware distribution In USENIXSecurity Symposium 2011

[5] N Christin Traveling the silk road A measurement analysis of alarge anonymous online marketplace In Proceedings of the 22ndinternational conference on World Wide Web pages 213ndash224 ACM2013

[6] N Falliere L OrsquoMurchu and E Chien W32Stuxnet dossierSymantec Whitepaper February 2011

[7] D GOODIN Stuxnet spawn infected kaspersky using stolen foxconndigital certificates Jun 2015

[8] InfoArmor GovRAT ndash Advanced Persistent ThreadsDigital certificates on sale in the underground httpwwwinfoarmorcomwp-contentuploads201604AdvancedPersistent Threats Code Signingpdf 2016

[9] C Kanich N Weaver D McCoy T Halvorson C KreibichK Levchenko V Paxson G M Voelker and S Savage Show methe money Characterizing spam-advertised revenue Usenix Security2011

[10] L Kessem Certificates-as-a-service code signing certs becomepopular cybercrime commodity httpssecurityintelligencecomcertificates-as-a-service-code-signing-certs-become-popular-cybercrime-commodity2015

[11] D Kim B J Kwon and T Dumitras Certified malware Measuringbreaches of trust in the windows code-signing pki In Proceedingsof the 2017 ACM SIGSAC Conference on Computer and Communi-cations Security CCS rsquo17 2017

[12] P Kotzias L Bilge and J Caballero Measuring pup prevalence andpup distribution through pay-per-install services In Proceedings ofthe USENIX Security Symposium 2016

[13] P Kotzias and J Caballero An analysis of pay-per-install economicsusing entity graphs WEIS 2017

[14] P Kotzias S Matic R Rivera and J Caballero Certified pup Abusein authenticode code signing In Proceedings of the 22Nd ACMSIGSAC Conference on Computer and Communications SecurityCCS rsquo15 pages 465ndash478 New York NY USA 2015 ACM

[15] K Kozak B J Kwon D Kim C Gates and T Dumitras Issuedfor Abuse Measuring the Underground Trade in Code Signing Cer-tificate ArXiv e-prints Mar 2018

[16] B J Kwon V Srinivas A Deshpande and T Dumitras Catchingworms trojan horses and pups Unsupervised detection of silentdelivery campaigns In 24th Annual Network and Distributed Sys-tem Security Symposium NDSS 2017 San Diego California USAFebruary 26 - March 1 2017 2017

[17] Microsoft Windows authenticode portable executable signature for-mat Mar 2008

[18] Microsoft Windows Defender SmartScreenhttpsdocsmicrosoftcomen-uswindowsthreat-protectionwindows-defender-smartscreenwindows-defender-smartscreen-overview 2017

[19] C Osborne Software code signing certificates worth morethan guns on the dark web httpwwwzdnetcomarticleillicit-certificates-worth-more-than-guns-on-the-dark-web Oct2017

[20] K L G Research and A Team The duqu 20 persistence moduleJun 2015

[21] M Sebastian R Rivera P Kotzias and J Caballero AVClass Atool for massive malware labeling RAID 2016

[22] K Soska and N Christin Measuring the longitudinal evolution of theonline anonymous marketplace ecosystem In Proceedings of the 24thUSENIX Conference on Security Symposium SECrsquo15 pages 33ndash48Berkeley CA USA 2015 USENIX Association

[23] Swiat Flame malware collision attack explained Jun 2012

[24] Symantec Corporation Symantec Extended Validation (EV)code signing certificate httpsknowledgesymanteccomsupportcodesigning-supportindexpage=contentampid=INFO1852ampactp=GETTING STARTED

[25] K Thomas J A E Crespo R Rasti J M Picod C PhillipsM Decoste C Sharp F Tirelo A Tofigh M Courteau L BallardR Shield N Jagpal M A Rajab P Mavrommatis N ProvosE Bursztein and D McCoy Investigating commercial pay-per-installand the distribution of unwanted software In 25th USENIX SecuritySymposium USENIX Security 16 Austin TX USA August 10-122016 pages 721ndash739 2016

[26] K Thomas D Huang D Wang E Bursztein C Grier T JHolt C Kruegel D McCoy S Savage and G Vigna Framingdependencies introduced by underground commoditization In WEIS2015

[27] VirusTotal Malware hunting httpswwwvirustotalcomhunting-overview

[28] M Wood Want My Autograph The Use and Abuse of DigitalSignatures by Malware Virus Bulletin Conference September 2010(September)1ndash8 2010

Appendix AList of Inspected Sites

In total 1 e-shop specializing in code-signing certifi-cates 28 forums 4 general markets and 6 link directorysites were inspected during the black market investigationndashall of these are listed below Dozens of sites specializedon other goods (such as stolen Credit Cards and Paypalaccounts) that were mentioned at the listed link directorysites were inspected as well but they are not listed here

15

since (1) as expected they did not provide any value orhints of code-signing certificates (2) this absence of valuewas not surprising (as opposed to general markets where weexpected to find anonymous code signing certificates) (3)their addresses were extracted from the link directory siteslisted here and (4) the space in this section is limited

Note that the Alphabay general market was not inspectedsince this research was carried out shortly after Alphabayhas been taken down by law enforcement All the URLsare given as they were valid during our research period(especially some of the darknet ones might not be validanymore)

A1 E-Shop with Anonymous Code-Signing Cer-tificates

codesigningguru

A2 Forums

hackforumsnetantichatru0daysufreehacksrunulledtobitcointalkorgsinisterlyhpcnamesearchenginesgurubinaryvisioncoilbitcointalkorgoffensivecommunitynetrussianelitews verifiedwsfsellbzforumzloybselitepvperscomcardersuprologicsutheccbzabusewithusv3rmillionnetCrutopnucardxbizcopsuwordcardingsupsh-worldrucrimezoneorgelitepvperscomforum

A3 General Markets

hansamkt2rr6nfg3onion (Hansa)lchudifyeqm4ldjjonion (Dream Market)traderouteilbgztonion (Trade Route)trdealmgn4uvm42gonion (theRealDeal)

A4 Link Directory Sites

deepdotwebcomtorlinkbgs6aabnsonion (Torlinks)underdj5ziov3ic7onion (UnderDir)onionsnjajzkhm5gonionzgrl6sghf5jh37zzoniontt3j2x4k5ycaa5ztonion

Appendix BForum and E-Shop Screenshots

Code signing certificates are often advertised on blackmarket forums Some posts include even rich graphics Anexample post is shown in Figure 10 and Figure 11 depictsa diagram of code signing certificate purchase from otherblack market post

Figure 10 Example post offering code signing certificateson the black market

The main page of the codesigning guru e-shop is shownin Figure 12 along with the page where Comodo codesigning certificates can be purchased in Figure 13

16

Figure 11 Code signing certificate purchase workflow illus-trated in a black market post

Figure 12 Main page of the codesigning guru e-shop

Appendix CAdvertisement Post Example

Irsquom selling anonymous code signing cert-ificates

Why do I need to sign my files- to avoid UAC warnings- to pass SmartScreen filter (Unknownpublisher)- to pass some AVs which are blockingany unsigned executables- to make your macros and VBA objectsmore trusted

What files can I sign- 32 and 64-bit applications (execab dll ocx msi xpi and xap)

Figure 13 Page for buying Comodo code signing certificatesin the codesigning guru e-shop

- Drivers (EV certificates only)- Java applications- Apple applications- VBA objects scripts and macros forMicrosoft Office doc xls and pptfiles

Certificate types- regular code signing certificate(class 3) yoursquoll get the archivewith PFX and a passwordRegular certs should gain a reputationbefore they pass SmartScreen filter(contact me for details)

- EV Code Signing cert yoursquoll get aUSB token with pre-installed certificatevia mail or courierEV certificates are the only ones thatyou can use for signing drivers forWin10Also they have a positive SmartScreenreputation out-of-the-box so no SSwarnings will appear

What you get- Unique and never used beforeCode Signing certificate from trustedCertificate Authority valid for 1 year- Free code signing tool with GUI(if needed)- All certificates are issued forreal companies

Disclaimer- I DO NOT resell certificates

17

from other sellers- I guarantee that your certificate isunique- I DO NOT offer one-time signing

Pricing- Comodo (most common) $350- Thawte (more trusted than Comodo)$600- EV code signing certs $3000- Free exchange if your cert becomesblacklisted by AVs you can get onere-issue for free- Escrow accepted- All payments by Bitcoin only

Current stock- Comodo 2- Thawte 0 (pre-order and get it in3-5 days)- EV Code Signing 0 (pre-order andget it in 2-3 weeks)Contacts- Jabber lte-mail address removedgt(use OTR please)- PM me

18

1 Introduction

2 Problem Statement




3 Data Collection




41 Vendors

42 Goods and Deals

43 Sales Volume







7 Discussion

8 Related work

9 Conclusions

References

Appendix A List of Inspected Sites

A1 E-Shop with Anonymous Code-Signing Certificates

A2 Forums

A3 General Markets


Appendix B Forum and E-Shop Screenshots

Appendix C Advertisement Post Example

overall activity of these vendors has been increasing in thefirst half of 2017 with one of the vendors setting up hisown e-shop in August 2017 We regularly collected stockinformation and analyzed the sales of this e-shop Duringthe 104 days of our observation period the e-shop obtaineda new certificate every two days on average and collected$16150 in revenue for selling these certificates A new codesigning certificate generally trades for a few hundred dollarsExtended Validation (EV) code signing certificates are alsooffered for a few thousand dollars each

To further investigate the impact of this undergroundtrade on signed malware we collect 14221 signed malwaresamples using the VirusTotal Hunting API [27] and extract1163 certificates from these samples By analyzing thesubset of samples that also carry a trusted timestamp wefind that around 45 of the abused certificates are used tosign malware within the first month after they were issuedThis pattern of abuse is consistent with a black marketthat frequently obtains new certificates and makes themavailable for general consumptionmdashas observed in our stockmeasurements for the code signing e-shopmdashbut would bedifficult to sustain by relying on signing keys stolen fromlegitimate publishers To further corroborate this connectionwe utilize several properties of the certificates to infer that 5certificates found in our signed malware samples had likelybeen purchased from the e-shop during our observationperiod

We analyze the relationships among the actors in thisunderground market by building a graph that connects pub-lisher identities with certificates and signed malware fami-lies We identify a large strongly connected component thatcontains most of the samples and mainly Russian publisheridentities with other components being generally small andwell separated The strong connectivity within the largecomponent suggests a degree of cooperation among thevarious malware developer teams and confidence in theblack market This cluster also exhibits the highest rate ofburning new certificates by using around 60 of them tosign malware within the first month after issuance

The emergence of this underground trade points to agrowth in the demand for code signing certificates from mal-ware authors The requests for certificates on undergroundforums and the marketing messages of the four certificatevendors suggest that a key factor driving this demand isthe need to bypass Microsoft Defender SmartScreen [18] acertificate reputation system built into Windows 10 In turnthis demand has created a market opportunity for specializedvendors who can obtain new certificates regularly and sellthem for immediate use

In summary this paper makes four contributions

bull We conduct an exploratory analysis of the under-ground trade in code signing certificates We illumi-nate its business model marketplaces and pattern oftransactions

bull We report evidence consistent with a shift in howmalware is signed as the demand for new certificatesleads to a growing prevalence of certificates issued

for abuse (rather than compromised certificates asreported in the prior work)

bull We perform a graph-based analysis of the attributesof signed malware and we infer new relationshipsamong malware developers

bull We use indirect evidence to infer that 5 certificatesfrom digitally signed malware found in the wild hadlikely been purchased from the underground market

We utilize our findings to draw lessons about the role ofthe certificate black market in the ecosystem of digitallysigned malware and we discuss concrete proposals forfacilitating the verification of publisher identities We releaseour data set of certificates extracted from signed malwaresamples at httpssignedmalwareorg

2 Problem Statement


Microsoft Authenticode [17] is a standard for digitallysigning Windows portable executables (PE) The digitalsignatures allow client platforms to identify the publisherwho developed the software and to ensure the integrityof the signed executables Each signature appended to anexecutable is accompanied by a code-signing certificatewhich binds the signing key to the publisherrsquos real iden-tity These certificates are issued and signed by CertificateAuthorities (CAs) who are trusted to verify the identitiesof the publishers who request code-signing certificates Eachcertificate includes a specification of the dates of issuanceand expiration To avoid the challenge of replacing binarieswith expired certificates in the field software publishersoften submit their binaries to Time-Stamping Authorities(TSAs) and receive unforgeable timestamps proving whenthe binary was signed Signed binaries that include suchtrusted timestamps are considered valid even after the certifi-cate expires Together these roles and procedures representthe Authenticode public key infrastructure (PKI) which isthe basis for establishing trust in Windows executables

Unlike the better studied Web PKI the AuthenticodePKI is opaque as compromised certificates cannot be dis-covered systematically through network scanning and thereis no official list of legitimate software publishers Thisfacilitates abuse allowing miscreants to obtain code sign-ing certificates and to produce valid digital signatures formalicious code Several mechanisms have been introducedto prevent this abuse

Revocation Requirements In 2016 the Certificate Author-ity Security Council (CASC) adopted a set of minimumrequirements for code signing certificates [3] According tothese guidelines a CA must promptly investigate and revokea code signing certificate after being notified that the certifi-cate has been used to sign malware This makes it difficultfor malware authors to reuse code signing certificates

2












3


3 Data Collection








4








Item Count









41 Vendors


5








D 2017-03-07 2017-08-13 2 4 7+ -









6





42 Goods and Deals






43 Sales Volume


7



















8











9














10










11









12







7 Discussion


8 Related work


13









9 Conclusions



14


Acknowledgment


References































15




codesigningguru

A2 Forums


A3 General Markets








16













17




18

1 Introduction

2 Problem Statement




3 Data Collection




41 Vendors

42 Goods and Deals

43 Sales Volume







7 Discussion

8 Related work

9 Conclusions

References



A2 Forums

A3 General Markets















3


3 Data Collection








4








Item Count









41 Vendors


5








D 2017-03-07 2017-08-13 2 4 7+ -









6





42 Goods and Deals






43 Sales Volume


7



















8











9














10










11









12







7 Discussion


8 Related work


13









9 Conclusions



14


Acknowledgment


References































15




codesigningguru

A2 Forums


A3 General Markets








16













17




18

1 Introduction

2 Problem Statement




3 Data Collection




41 Vendors

42 Goods and Deals

43 Sales Volume







7 Discussion

8 Related work

9 Conclusions

References



A2 Forums

A3 General Markets





3 Data Collection








4








Item Count









41 Vendors


5








D 2017-03-07 2017-08-13 2 4 7+ -









6





42 Goods and Deals






43 Sales Volume


7



















8











9














10










11









12







7 Discussion


8 Related work


13









9 Conclusions



14


Acknowledgment


References































15




codesigningguru

A2 Forums


A3 General Markets








16













17




18

1 Introduction

2 Problem Statement




3 Data Collection




41 Vendors

42 Goods and Deals

43 Sales Volume







7 Discussion

8 Related work

9 Conclusions

References



A2 Forums

A3 General Markets











Item Count









41 Vendors


5








D 2017-03-07 2017-08-13 2 4 7+ -









6





42 Goods and Deals






43 Sales Volume


7



















8











9














10










11









12







7 Discussion


8 Related work


13









9 Conclusions



14


Acknowledgment


References































15




codesigningguru

A2 Forums


A3 General Markets








16













17




18

1 Introduction

2 Problem Statement




3 Data Collection




41 Vendors

42 Goods and Deals

43 Sales Volume







7 Discussion

8 Related work

9 Conclusions

References



A2 Forums

A3 General Markets











D 2017-03-07 2017-08-13 2 4 7+ -









6





42 Goods and Deals






43 Sales Volume


7



















8











9














10










11









12







7 Discussion


8 Related work


13









9 Conclusions



14


Acknowledgment


References































15




codesigningguru

A2 Forums


A3 General Markets








16













17




18

1 Introduction

2 Problem Statement




3 Data Collection




41 Vendors

42 Goods and Deals

43 Sales Volume







7 Discussion

8 Related work

9 Conclusions

References



A2 Forums

A3 General Markets








42 Goods and Deals






43 Sales Volume


7



















8











9














10










11









12







7 Discussion


8 Related work


13









9 Conclusions



14


Acknowledgment


References































15




codesigningguru

A2 Forums


A3 General Markets








16













17




18

1 Introduction

2 Problem Statement




3 Data Collection




41 Vendors

42 Goods and Deals

43 Sales Volume







7 Discussion

8 Related work

9 Conclusions

References



A2 Forums

A3 General Markets






















8











9














10










11









12







7 Discussion


8 Related work


13









9 Conclusions



14


Acknowledgment


References































15




codesigningguru

A2 Forums


A3 General Markets








16













17




18

1 Introduction

2 Problem Statement




3 Data Collection




41 Vendors

42 Goods and Deals

43 Sales Volume







7 Discussion

8 Related work

9 Conclusions

References



A2 Forums

A3 General Markets














9














10










11









12







7 Discussion


8 Related work


13









9 Conclusions



14


Acknowledgment


References































15




codesigningguru

A2 Forums


A3 General Markets








16













17




18

1 Introduction

2 Problem Statement




3 Data Collection




41 Vendors

42 Goods and Deals

43 Sales Volume







7 Discussion

8 Related work

9 Conclusions

References



A2 Forums

A3 General Markets

















10










11









12







7 Discussion


8 Related work


13









9 Conclusions



14


Acknowledgment


References































15




codesigningguru

A2 Forums


A3 General Markets








16













17




18

1 Introduction

2 Problem Statement




3 Data Collection




41 Vendors

42 Goods and Deals

43 Sales Volume







7 Discussion

8 Related work

9 Conclusions

References



A2 Forums

A3 General Markets













11









12







7 Discussion


8 Related work


13









9 Conclusions



14


Acknowledgment


References































15




codesigningguru

A2 Forums


A3 General Markets








16













17




18

1 Introduction

2 Problem Statement




3 Data Collection




41 Vendors

42 Goods and Deals

43 Sales Volume







7 Discussion

8 Related work

9 Conclusions

References



A2 Forums

A3 General Markets












12







7 Discussion


8 Related work


13









9 Conclusions



14


Acknowledgment


References































15




codesigningguru

A2 Forums


A3 General Markets








16













17




18

1 Introduction

2 Problem Statement




3 Data Collection




41 Vendors

42 Goods and Deals

43 Sales Volume







7 Discussion

8 Related work

9 Conclusions

References



A2 Forums

A3 General Markets










7 Discussion


8 Related work


13









9 Conclusions



14


Acknowledgment


References































15




codesigningguru

A2 Forums


A3 General Markets








16













17




18

1 Introduction

2 Problem Statement




3 Data Collection




41 Vendors

42 Goods and Deals

43 Sales Volume







7 Discussion

8 Related work

9 Conclusions

References



A2 Forums

A3 General Markets












9 Conclusions



14


Acknowledgment


References































15




codesigningguru

A2 Forums


A3 General Markets








16













17




18

1 Introduction

2 Problem Statement




3 Data Collection




41 Vendors

42 Goods and Deals

43 Sales Volume







7 Discussion

8 Related work

9 Conclusions

References



A2 Forums

A3 General Markets





Acknowledgment


References































15




codesigningguru

A2 Forums


A3 General Markets








16













17




18

1 Introduction

2 Problem Statement




3 Data Collection




41 Vendors

42 Goods and Deals

43 Sales Volume







7 Discussion

8 Related work

9 Conclusions

References



A2 Forums

A3 General Markets







codesigningguru

A2 Forums


A3 General Markets








16













17




18

1 Introduction

2 Problem Statement




3 Data Collection




41 Vendors

42 Goods and Deals

43 Sales Volume







7 Discussion

8 Related work

9 Conclusions

References



A2 Forums

A3 General Markets
















17




18

1 Introduction

2 Problem Statement




3 Data Collection




41 Vendors

42 Goods and Deals

43 Sales Volume







7 Discussion

8 Related work

9 Conclusions

References



A2 Forums

A3 General Markets







18

1 Introduction

2 Problem Statement




3 Data Collection




41 Vendors

42 Goods and Deals

43 Sales Volume







7 Discussion

8 Related work

9 Conclusions

References



A2 Forums

A3 General Markets




Issued for Abuse: Measuring the Underground Trade in Code ... · $16,150 in revenue for selling...

Documents

Transcript of Issued for Abuse: Measuring the Underground Trade in Code ... · $16,150 in revenue for selling...