AccD: A Compiler-based Framework forAccelerating Distance-related Algorithms on

CPU-FPGA PlatformsYuke Wang1, Boyuan Feng1, Gushu Li2, Lei Deng2, Yuan Xie2, and Yufei Ding1

1Department of Computer Science2Department of Electrical and Computer Engineering

1{yuke_wang,boyuan,yufeiding}@cs.ucsb.edu2{gushuli,leideng,yuanxie}@ece.ucsb.edu

University of California, Santa Barbara

Abstract—As a promising solution to boost the performanceof distance-related algorithms (e.g., K-means and KNN), FPGA-based acceleration attracts lots of attention, but also comeswith numerous challenges. In this work, we propose AccD,a compiler-based framework for accelerating distance-relatedalgorithms on CPU-FPGA platforms. Specifically, AccD providesa Domain-specific Language to unify distance-related algorithmseffectively, and an optimizing compiler to reconcile the benefitsfrom both the algorithmic optimization on the CPU and thehardware acceleration on the FPGA. The output of AccD is ahigh-performance and power-efficient design that can be easilysynthesized and deployed on mainstream CPU-FPGA platforms.Intensive experiments show that AccD designs achieve 31.42×speedup and 99.63× better energy efficiency on average overstandard CPU-based implementations.

I. INTRODUCTION

Distance-related algorithm (e.g., K-means [1], KNN [2],and N-body Simulation [3]) plays a vital role in many do-mains, including machine learning, computational physics, etc.However, these algorithms often come with high computationcomplexity, leading to poor performance and limited applica-bility. To improve their performance, FPGA-based accelerationgains lots of interests from both industry and research field,given its great performance and energy-efficiency. However,accelerating distance-related algorithms on FPGAs requiresnon-trivial efforts, including the hardware expertise, time andmonetary cost. While existing works try to ease this process,they inevitably fall in short in one of the following aspects.

Rely on problem-specific design and optimization whilemissing effective generalization. There is no such unifiedabstraction to formalize the definition and optimization ofdistance algorithms systematically. Most of the previous hard-ware designs and optimizations [4–7] are heavily coded for aspecific algorithm (e.g., K-means), which can not be sharedwith different distance-related algorithms. Moreover, these”hard-coded” strategies could also fail to catch up with theever-changing upper-level algorithmic optimizations and theunderlying hardware settings, which could result in a largecost of re-design and re-implementation during the designevolvement.

Lack of algorithm-hardware co-design. Previous algorith-mic [8, 9] and hardware optimizations [4–7, 10] are usuallyapplied separately instead of being combined collaboratively.Existing algorithmic optimizations, most of which are basedon Triangle Inequality (TI) [8, 9, 11, 12], are crafted forsequential-based CPU. Despite removing a large number ofdistance computations, they also incur high computation ir-regularity and memory overhead. Therefore, directly applyingthese algorithmic optimizations to massively parallel platformswithout taking appropriate hardware-aware adaption couldlead to inferior performance.

Count on FPGAs as the only source of acceleration.Previous works [4, 7, 13–16] place the whole algorithm onthe FPGA accelerator without considering the assists from thecomputing resource on the host CPU. As a result, their designsare usually limited by the on-chip memory and computingelements, and cannot fully exploit the power of the FPGA.Moreover, they miss the full performance benefits from theheterogeneous computing paradigm, such as using the CPUfor complex logic and control operations while offloading thecompute-intensive tasks to the FPGA.

Lack of well-structured design workflow. Previousworks [5–7, 13, 15] follow the traditional way of hardwareimplementation and require intensive user involvement inhardware design, implementation, and extra manual tuningprocess, which usually takes long development-to-validationcycles. Also, the problem-specific strategy leads to a case-by-case design process, which cannot be widely applied tohandle different problem settings. To this end, we presenta compiler-based optimization framework, AccD, to auto-matically accelerate distance-related algorithms on the CPU-FPGA platform (shown in Figure 1). First, AccD provides aDistance-related Domain-Specific Language (DDSL) as aproblem-independent abstraction to unify the description andoptimization of various distance-related algorithms. With theassist of the DDSL, end-user can easily create highly-efficientCPU-FPGA designs by only focusing on high-level problemspecification without touching the algorithmic optimization orhardware implementation.

arX

iv:1

908.

1178

1v1

[cs

.DC

] 2

6 A

ug 2

019

DD

SL Alg

orith

m

Descrip

tion

OpenCL KernelCompiler

GCC Compiler

OpenCLLibarary

Linking

.cppsource

.clsource

PCIe

AccD Compiler

Design Space Explorer

Hardwre Optimization

Kernel

Memory

Hardwre Optimization

Kernel

Memory

AlgorithmicOptimization

StrategySelection

GTI

AlgorithmicOptimization

StrategySelection

GTI

DD

SL Algorith

m D

escriptio

n OpenCL Kernel Compiler

GCC Compiler

OpenCL Libarary

Linking

Host Source

Kernel Source

PCIe

AccD Compiler

Design Space Explorer (DSE)

Algorithmic Optimization

Generalized Triangle-Inequality (GTI)

Hardware Optimization

Distance Computation Kernel

Memory Layout

Executing

Deploying

• Two-landmark Bound• Trace-based Bound• Group-level Bound

Fig. 1: AccD Overview.

Second, AccD offers a novel algorithmic-hardware co-optimization scheme to reconcile the acceleration from bothsides. At the algorithmic level, AccD incorporates a novelGeneralized Triangle Inequality (GTI) optimization to elimi-nate unnecessary distance computations, while maintaining thecomputation regularity to a large extent. At the hardware level,AccD employs a specialized data layout to enforce memorycoalescing and an optimized distance computation kernel toaccelerate the distance computations on the FPGA.

Third, AccD leverages both the host and accelerator side ofthe CPU-FPGA heterogeneous system for acceleration. Inparticular, AccD distributes the algorithm-level optimization(e.g., data grouping and distance computation filtering) toCPU, which consists of complex operations and executiondependency, but lacks pipeline and parallelism. On the otherhand, AccD assigns hardware-level acceleration (e.g., distancecomputations) to the FPGA, which is composed of simple andvectorizable operations. Such mapping successfully capitalizesthe benefit of CPU for managing control-intensive tasks andthe advantage of FPGA for accelerating computation-intensiveworkloads.

Lastly, AccD compiler integrates an intelligent DesignSpace Explorer (DSE) to pinpoint the ”optimal” design fordifferent problem settings. In general, there is no existing ”onesize fits all” solution: the best configuration for algorithmic andhardware optimization would differ across different distance-related algorithms or different inputs of the same distance-related algorithm. To produce a high-quality optimizationconfiguration automatically and efficiently, DSE combinesthe design modeling (performance and resource) and GeneticAlgorithm to facilitate the design space search.

Overall, our contributions are:

• We propose the first optimization framework that canautomatically optimize and generate high-performanceand power-efficient designs of distance-related algorithmson CPU-FPGA heterogeneous computing platforms.

• We develop a Domain-specific Language, DDSL, to unifydifferent distance-related algorithms in an effective andsuccinct manner, laying the foundation for general opti-mizations across different problems.

• We build an optimizing compiler for the DDSL, whichautomatically reconciles the benefits from both the algo-rithmic optimization on CPU and hardware accelerationon FPGA.

• Intensive experiments on several popular algorithmsacross a wide spectrum of datasets show that AccD-generated CPU-FPGA designs could achieve 31.42×speedup and 99.63× better energy-efficiency on averagecompared with standard CPU-based implementations.

II. RELATED WORKPrevious research accelerates distance-related algorithms in

two aspects: Algorithmic Optimization and Hardware Acceler-ation. More details are discussed in the following subsections.

A. Algorithmic Optimization

From the algorithmic standpoint, previous research high-lights two optimizations. The first one is KD-tree based opti-mization [17–21], which relies on storing points in special datastructures to enable nearest neighbor search without computingdistances to all target points. These methods often deliver3× ∼ 6× performance improvement [17–21] compared withthe unoptimized versions in low dimensional space, while suf-fering from a serious performance degradation when handlinglarge datasets with high dimension (d ≥ 20) due to theirexponentially-increased memory and computation overhead.

The second one is TI based optimization [8, 9, 11, 12],which aims at replacing computation-expensive distance com-putations with cheaper bound computations, demonstrates itsflexibility and scalability. It can not only reduce the compu-tation complexity at different levels of granularity but is alsomore adaptive and robust to the datasets with a wide rangeof size and dimension. However, most existing works focuson one specific algorithm (e.g., KNN [12], K-means [8, 9],etc.), which lack extensibility and generality across differentdistance-related problems. An exception is a recent work,TOP [11], which builds a unified framework to optimizevarious distance-related problems with pure TI optimizationon CPUs. Our work shares a similar high-level motivation withtheir work, but targets at a more challenging scenario: algorith-mic and hardware co-optimization on CPU-FPGA platforms.

B. Hardware Acceleration

From the hardware perspective, several FPGA acceleratordesigns have been proposed, but still suffer from some majorlimitations.

First, previous FPGA designs are generally built for spe-cific distance-related algorithm and hardware. For example,works from [4–6] target on KNN FPGA acceleration, while

researches from [7, 13, 14] focus on K-means. Moreover,previous designs [4, 5] usually assume that dataset can be fullyfit into the FPGA on-chip memory, and they are only evaluatedon a limited number of small datasets, for example, in [5], K-means acceleration is evaluated on a micro-array dataset withonly 2,905 points. These designs often encounter portabilityissues when transferring to different settings. Besides, these”hard-coded” designs and optimizations create difficulties for afair comparison among different designs, which hamper futurestudies in this direction.

The second problem with previous works is that theyfail to incorporate algorithmic optimizations in the hardwaredesign. For example, works from [4, 6, 7, 13], directly portthe standard K-means and KNN algorithms to FPGA, andonly apply hardware-level optimization. One exception is arecent work [22], which promotes to combine TI optimizationand FPGA acceleration for K-means. It gives a considerablespeedup compared to state-of-the-art methods, showcasingthe great opportunity of applying algorithm-hardware co-optimization. Nevertheless, this idea is far from well-explored,possibly because it requires the domain knowledge and exper-tise from both the algorithm and hardware to combine both ofthem effectively.

In addition, previous works largely focus on the traditionalhardware design flow, which requires a long implementationcycle and huge manual efforts. For example, works from [4,6, 10, 15, 16, 23–25] build the design based on VHDL/Ver-ilog design flow, which requires hardware expertise and overmonths of arduous development. In contrast, our AccD designflow brings significant advantages of programmability andflexibility due to its high-level OpenCL-based programmingmodel, which minimizes the user involvement in the tedioushardware design process.

III. DISTANCE-RELATED ALGORITHM DOMAIN-SPECIFICLANGUAGE (DDSL)

Distance-related algorithms share commonalities across dif-ferent application domains and scenarios, even though theylook different in their high-level algorithmic description.Therefore, it is possible to generalize these distance-relatedalgorithms. AccD framework defines a DDSL, which pro-vides a high-level programming interface to describe distance-related algorithms in a unified manner. Unlike the API-basedprogramming interface used in the TOP framework [11],DDSL is built on C-like language and provides more flexibilityin low-level control and performance tuning support, which iscrucial for FPGA accelerator design.

Specifically, DDSL utilizes several constructs to describethe basic components (Definition, Operation, and Control) ofthe distance-related algorithms, and also identify the potentialparallelism and pipeline opportunities during the design time.We detail these constructs in the following part of this section.

A. Data Construct

Data construct is a basic Definition Construct. It leveragesDSet primitive to indicate the name of the data variable,

and the DType primitive to notate the type characters of thedefined variable. Data construct serves as the basis for AccDcompiler to understand the algorithm description input, such asthe data points that are used in the distance-related algorithms.An example of data constructs is shown in the code below,where we define the variable and dataset using DDSL dataconstruct.

/* Define a single variable */DVar [setName] DType [Optional_Initial_Value];/* Define the matrix of dataset */DSet [setName] DType [size] [dim];

In most distance-related algorithms, the dataset can bedefined as the source set and the target set. For example, in K-means, the source set is the set of data points, and the targetset is the set of clusters. Currently, AccD supports severaldata types including int (32-bit), float (32-bit), double (64-bit) based on the users’ requests, algorithm performance, andaccuracy trade-offs.

B. Distance Compute Construct

Distance computation is the core Operation Construct fordistance-related algorithms, which measures the exact distancebetween two different data points. This construct requiresseveral fields, including data dimensionality, distance metrics,and weight matrix (if weighted distance is specified).

AccD_Comp_Dist(Input p1, Input p2, OutputdisMat, Output idMat, Dim dim, Met mtr,Weg mat)

p1, p2 Input data matrix. (n1 × d, n2 × d)disMat Output distance matrix. (n1 × n2)idMat Output id matrix. (n1 × n2)dim Dimensionality of input data point.mtr Distance metric:(Weighted|Unweighted)mat Weight matrix: Used for weighted distance (1× d)

TABLE I: Distance Compute Construct Parameters.C. Distance Selection Construct

Distance selection construct is an Operation Construct fordistance value selection and it returns the Top-K smallest orlargest distances and their corresponding points ID numberfrom the provided distance and ID list. This construct helpsAccD compiler to understand the distances of users’ interests.

AccD_Dist_Select(Input distMat, Input idMat,Output TopKMat, Range ran, Scope scp)

TopKMat Top-K id matrix (n1 × k)ran Scalar value of K (e.g., K-means, KNN) or

distance threshold (e.g., N-body Simulation)scp Top-K (smallest|largest) values

TABLE II: Distance Selection Construct Parameters.

D. Data Update ConstructData update construct is an Operation Construct for

updating the data points based on the results from the priorconstructs. For example, K-means updates the cluster centers

by averaging the positions of the points inside. This constructrequires the variable to be updated and additional informationto finish this update, such as the point-to-cluster distances. Thestatus of this data update will be returned after the completionof all its inside operations. The status variable is to tell whetherthe data update makes a difference or not.

AccD_Update(Update var, Input p1 ,..., Inputpm, Status s)

upVar Input data/dataset to be updatedp1, ..., pm Additional information used in update

S Status of update operation.

TABLE III: Data Update Construct Parameters.E. Iteration Construct

Iteration construct is a top-level Control Construct. It isused to describe the distance-related algorithms that requireiteration, such as K-means. Iteration construct requires usersto provide either the maximum number of iteration or otherexit condition.

AccD_Iter(maxIterNum|exitCond){subConstruct sc1;subConstruct sc2;...subConstruct scn;

}

F. Example: K-meansTo show the expressiveness of DDSL, we take K-means as

an example. From the code shown below, with no more than20 lines of code, DDSL can capture the key components ofuser-defined K-means algorithm, which is essential for AccDcompiler to generate designs for CPU-FPGA platforms.

DVar K int 10;DVar D int 20;DVar psize int 1400;DVar csize int 200;DSet pSet float psize D;DSet cSet float csize D;DSet distMat float psize csize;DSet idMat int psize csize;DSet pkMat int psize K;AccD_Iter(S){

S = false;/* Compute the inter-dataset distances */AccD_Comp_Dist(pSet, cSet, distMat, idMat,

D, "Unweighted L1", 0);/* Select the distances of interests */AccD_Dist_Select(distMat, idMat, K,

"smallest", pkMat);/* Update the cluster center */AccD_Update(cSet, pSet, pkMat, S)

}

IV. ALGORITHM OPTIMIZATION

This section explains a novel TI optimizations tailoredfor CPU-FPGA platforms. TI has been used for optimizingdistance-related problems, but is often on the sequential pro-cessing systems. Our design features an innovative way of

applying TI to obtain low-overhead distance bounds for un-necessary distance computation elimination while maintainingthe computation regularity to ease the hardware accelerationon FPGAs.

A. TI in Distance-related Algorithm

As a simple but powerful mathematical concept, TI hasbeen used to optimize the distance-related algorithm. Figure 2agives an illustration. It states that d(A,B) ≤ d(A,Lref ) +d(Lref , B), where d(A,B) represents the distance betweenpoint A and B in some metrics (e.g., Euclidean distance).The assistant point Lref is a landmark point for reference.Directly from the definition, we could compute both the lowerbound (lb(A,B)) and upper bound (ub(A,B)) of the distancebetween two points A and B. This is the standard and mostcommon usage of TI for deriving bounds of distance.

In general, bounds can be used as a substitute for the exactdistances in the distance-related data analysis. Take N-bodysimulation as an example. It requires to find target pointsthat are within R (the radius) from each given query point.Suppose we get the lb(A,B) = 10 and 10 > R, then weare 100% confident that source point A is not within R ofquery point B. As a result, there is no need to computethe exact distance between point A and B. Otherwise, theexact distance computation will still be carried out for directcomparison. While many previous researches [5, 8, 9, 26, 27]gain success in directly porting the above point-based TI tooptimize distance-related algorithms, they usually suffer frommemory overhead and computations irregularity, which resultin inferior performance.B. Generalized Triangle Inequality (GTI)

AccD uses a novel Generalized TI (GTI) to remove re-dundant distance computation. It generalizes the traditionalpoint-based TI while significantly reducing the overhead ofbound computations. The traditional point-based TI focuseson tighter bound (more closer to the exact distance) to removemore distance computations, but it induces the extra boundcomputations, which could become the new performance bot-tleneck even after many distance calculations being removed.In contrast, GTI strikes a good balance between distancecomputation elimination and bound computation overhead. Inparticular, AccD highlights GTI from three perspectives: Two-landmark bound computation, Trace-based bound computa-tion, and Group-level bound computation.

a) Two-landmark Bound Computation: Two-landmarkscheme aims at reducing the bound computation througheffective distance reuse. In this case, the distance boundbetween two points can be measured through two landmarks asthe reference points. As illustrated in Figure 2b, the distancebound between point A and B can be computed based ond(A,Aref ), d(B,Bref ) and d(Aref , Bref ) through Equation1, where Aref and Bref are the landmark points for point Aand B, correspondingly.

lb(A,B) ≥ d(Aref , Bref )− d(A,Aref )− d(B,Bref )ub(A,B) ≤ d(Aref , Bref ) + d(A,Aref ) + d(B,Bref )

(1)

Lref

A B(a)

A B’(c)

B

(f)

X

PointPoint

Point Old Pos.Point Old Pos.

X LandmarkDistance BoundDistance Bound

DistanceDistance

X X

X

X X X

X

X

A B

X X

(e)

A B

X X

(e)

A’

A B

B’(d)

A’

A B

B’(d)

A

Aref Bref

B(b)

XXA

Aref Bref

B(b)

XX

Fig. 2: TI Optimization.

One representative application scenario of Two-landmarkbound computation is KNN-join, where two disjoint sets oflandmarks are selected for the query and target point set.In this case, much fewer bound computations are requiredcompared with the one-landmark case (shown in Figure 2a).This can also be validated through a simple calculation.Assuming in KNN-join, we have m query points, n targetpoints, zqry query landmarks, and ztrg target landmarks. Also,we have zqry

where G ∈ {A,B}. If we have d(c,G) − dmax(G,G′) >d(c, d) + dmax(d, d

′), it is impossible that the points insidethe group A′ and B′ can become the closest point of c in thecurrent iteration. Therefore, the distance computation betweenthe point c and all points inside these groups can be safelyavoided.

In addition to distance saving, group-level bound computa-tion offers another two benefits to facilitate the underlyinghardware acceleration. First, the computation regularity onthe remaining distance computation becomes higher comparedwith the point-level bound computation. Since points insideeach group will share the commonality in computation, whichfacilitates the parallelization for acceleration. For example,point-level bound computation usually results in a large di-vergence of distance computation among different points, asshown in Figure 3a, which is a killer of parallelization andpipelining. However, in group-level bound computation, pointsinside the same source group will always maintain the samegroups of target points for distance computation, as shown inFigure 3b. Second, group-level bound computation brings the

Source Points

Point Target Point

sp1 tp1, tp2, tp4

tp3, tp4

tp4

sp2

sp3

tp2, tp3 sp4

SG Target Group

SG1 TG1, TG2

TG2SG2

TG

1T

G 2

SG 1

SG 2

SG 1

Target Points

Source Points

Target Points

tp1

tp2

tp3

tp4

sp1

sp2

sp3

sp4

tp1

tp2

tp3

tp4

sp1

sp2

sp3

sp4

(a)

(b)

Fig. 3: Bound Computation at (a) Point-level, (b) Group-level.

benefit of reducing memory overhead. Assuming we have msource points, n target points, zsrc source groups, and ztrgtarget groups. The memory overhead of maintaining distancebounds is O(m × n) in the point-level bound computationcase. However, in the group-level bound computation case,we only have to maintain distance bounds among groups, andthe memory overhead is O(zsrc×ztrg), where zsrc

continuous memory space within the same memory bank,as illustrated in Figure 5b. This strategy can largely benefitmemory coalescing and external memory bandwidth whileminimizing the access contention, since points inside the samebank can be accessed efficiently and points inside differentbanks can be accessed in parallel.

Group PointsGrp1 3, 8, 9Grp2 5, 6, 7Grp3 1, 2, 4

... ...

(a)

Grp 2

Grp 3

(a) (b)

Grp 1

Grp 1

Grp 3

Grp 2

Ba

nk 1

Ba

nk 2

57

Ba

nk 3

39

14

62

8

12

34

56

78

9

(b)

Grp 2

Grp 3

(a) (b)

Grp 1

Grp 1

Grp 3

Grp 2

Ba

nk 1

Ba

nk 2

57

Ba

nk 3

39

14

62

8

12

34

56

78

9

(c)

Fig. 5: (a) Group-point mapping; (b) Non-aligned intra-groupmemory; (c) Aligned intra-group memory.

B. Distance Computation Kernel

Distance computation takes the major time complexity indistance-related algorithms. In AccD, after TI filtering onCPU, the remaining distance computations are accelerated onFPGA. Points involved in the remaining distance computationsare organized into two sets: source set and target set, whichcan be organized as two matrices, MatA (m× d) and MatB(n × d), respectively, where each row of these matricesrepresents a point with d dimension. The distance computationbetween MatA and MatB can be decomposed into threeparts, as shown in Equation 4,

(MatA −MatB)2 =Mat2A − 2 ∗MatA ·MatB +Mat2B (4)

where Mat2A or Mat2B only takes the complexity of O(m×d)

and O(n×d), while MatA×MatB takes O(m×n×d2), whichdominates the overall computation complexity. AccD spots anefficient way of accelerating MatA ·MatB through highly-efficient matrix-matrix multiplication, which can benefit thehardware implementation on FPGA.

Source Set (A)

Target Set (B)Distance Results

Targ

et M

atri

x

MM

Resu

lts

Sou

rce

Mat

rix + -

x2

Dista

nce

Resu

lts

Row Square

Sum

+ -x2

Row Square

Sum

OpenCLKernel

On

-Ch

ipLo

cal M

emory

...

...

Src matrix

Trg matrix

KernelWork Group

Target

MM

Re

sults

Sou

rce

+ -x2

Dista

nce R

esults

RSS

RSS

OpenCL Kernel

Kernel Block

Fig. 6: AccD Matrix-based Distance Computation.

The overall computation process can be described as Fig-ure 6, the source (MatA) and target set (MatB) Row-wise

Square Sum (RSS) is pre-computed through in a fully-parallelmanner. And the vector multiplication between each sourceand target point is mapped to an OpenCL kernel thread for afine-grained parallelization. Moreover, a block of threads, ashighlighted in the ”red” square box of Figure 6, is the kernelthread workgroup, which can share a part of the source andtarget points to increase the on-chip data locality. Based onthis kernel organization, AccD hardware architectural designoffers several tunable hyperparameters for performance andresource trade-off: the size of kernel block, the number ofparallel pipeline in each kernel block, etc. To efficiently findthe ”optimal” parameters that can maximize overall perfor-mance while respecting the constraints, we harness the AccDexplorer for efficient design space search, which is detailed inSection VI-B.

VI. ACCD COMPILER

In this section, we detail AccD compiler in two aspects: de-sign parameters and constraints, and design space exploration.

A. Design Parameters and Constraints

AccD uses a parameterized design strategy for better designflexibility and efficiency. It takes the design parameters andconstraints from algorithm and hardware to explore and locatethe ”optimal” design point tailored for the specific applicationscenario. At the algorithm level, the number of groups affectsdistance computation filtering performance. At the hardwarelevel, there are three parameters: 1) Size of computationblock, which decides the size of data shared by a groupof computing elements; 2) SIMD factor, which decides thenumber of computing elements inside each computation block;3) Unroll factor, which tells the degree of parallelization ineach single distance computation. In addition, there are severalhardware constraints, such as the on-chip memory size, thenumber of logic units, and the number of registers. All ofthese parameters and constraints are included in our analyticalmodel for design exploration.

B. Design Space Exploration

Finding the best combination of design configurations (aset of hyper-parameters) under the given constraints requiresnon-trivial efforts in the design space search. Therefore, weincorporate an AccD explorer in our compiler framework forefficient design space exploration (Figure 7). AccD explorertakes a set of raw configurations (hyper-parameters) as theinitial input, and generates the optimal configuration as theoutput through several iterations of the design configurationoptimization process. In particular, AccD explorer consists ofthree major phases: Configuration Generation and Selection,Performance and Resource Modeling, Constraints Validation.

a) Configuration Generation and Selection: The func-tionality of this phase depends on its input. There are two kindsof inputs: If the input is from the initial configurations, thisphase will directly feed these configurations to the modelingphase for performance and resource evaluation; If the input is

AccD Compiler

Data-AlgorithmAbstraction

Grouping Strategy Selection

Filtering Strategy Selection

Data Spec.

Oper. Spec.

AccD Explorer

Intel OpenCL

SDK

Grp.Strategy

Filt. Strategy

DDSL Algorithm

Description

Configuration Generation &

Selection

Constraints Validation

Hardware Micro-Benchmark

Design ConstraintsGenetic Algorithm

Modeling

Resource

Performance

Modeling

Resource

Performance

Initial C

on

figuratio

ns

Op

timize

d

Co

nfigu

ration

s

Constrains/Performance Unsatisfied

AccD Explorer

Fig. 7: AccD Explorer.

the result from the constraints validation in the last iteration,this phase will leverage the genetic algorithm to crossoverthe ”premium” configurations kept from the last iteration, andgenerate a new set of configurations for the modeling phase.

b) Performance Modeling: Performance modeling mea-sures the design latency and bandwidth requirement basedon the input design configurations. We formulate the designlatency by using Equation 5,

Latency = Latencyfilt + Latencycomp (5)

where Latencyfilt and Latencycomp are the time of theGTI filtering process and remaining distance computations,respectively. And they can be calculated as Equation 6,

Latencyfilt =ntrg grp × nsrc grp × srcsize × trgsize × d

niteration

Latencycomp =srcsize × trgsize × ratiosave × dblk2 × frequency × unroll × simd

(6)

where nsrc grp and ntrg grp are the number of groups forsource and target points, respectively; srcsize and trgsize arethe number of points inside source and target set, respectively;d is the data dimensionality; niteration is the number ofgrouping iteration; blk is the size of computation kernel block;frequency is the FPGA design clock frequency; unroll isthe distance computation unroll factor; simd is the numberof parallel worker threads inside each computation block;ratiosave is the distance saving ratio through GTI filtering(Equation 7),

ratiosave =niteration

α×

√srcsize × trgsizensrc grp × ntrg grp

(7)

where the α the density of points distribution. This formulaalso tells that increasing of number of iterations and numberof points inside each group would improve the performanceof GTI filtering performance. Also, the increase of pointsdistribution density α, (i.e. points are closer to each other)will decrease the GTI filtering performance.

To get the required bandwidth BW of the current design,we leverage Equation 8,

BW =(srcsize + trgsize)× d× sizedata type

Latency(8)

where the sizedata type can be either 32-bit for int and floator 64-bit for double.

c) Resource Modeling: Directly measuring the hardwareresource usage of the accelerator design from high-level algo-rithm description is challenging because of the hidden trans-formation and optimization in hardware design suite. However,AccD uses a micro-benchmark based methodology to measurethe hardware resource usage by analytical modeling. Themajor hardware resource consumption of AccD comes fromthe distance computation kernel, which depends on severaldesign factors, including the kernel block size, the number ofSIMD workers, etc.

In AccD resource analytical model, the design factors areclassified into two categories: dataset-dependent and dataset-independent factors. The main idea behind the AccD resourcemodeling is to get the exact hardware resource consumptionstatistics through micro-benchmark on the hardware designswith different dataset-independent factors. For example, wecan benchmark a single distance computation kernel with dif-ferent sizes of computation block to get its resource statistics.Since this factor is dataset-independent, which can be decidedbefore knowing the dataset details. However, to estimatethe resource consumption for datasets with different sizesand dimensionalities, AccD leverages the formula-based ap-proach to estimate the overall hardware resource consumption(Equation 9), which combines online information (e.g., kernelorganization, and dataset properties) and offline information(e.g., miro-benchmark statistics).

Resourceest = Resourcesingle × ceil(srcsizeblk

)× ceil( trgsizeblk

)

(9)where the types of Resource can be on-chip memory, com-puting units, and logic operation units; Resourceest is theestimated overall usage of a certain type resource for theoverall design; Resourcesingle is the usage of a certain typeof resource for only one distance computation kernel block.

d) Constraints Validation: Constraints validation is thethird phase of AccD explorer, which checks whether thedesign resources consumption of a given configuration iswithin the budget of the given hardware platform. The inputof this phase are the resource estimation results from resourcemodeling step. The design constraint inequalities are listed inEquation 10, which includes Mem (the size of on-chip mem-ory), BW (the bandwidth of data communication betweenexternal memory and on-chip memory), Computing Unit(the number of computing units) and Logic Unit (the numberof logic units):

BW ≤ BWmax,Mem ≤Memmax,

Computing Unit ≤ Computing Unitmax,Logic Unit ≤ Logic Unitmax

(10)

Constraints validation phase will also discard the con-figurations that cannot match the design performance andconstraints, and only keep the ”well-performed” configurationsfor further optimization in the next iteration. The constraintsvalidation phase will also record the modeling information ofthe best configuration statistics in the last iteration, which will

be used to terminate the optimization process if the modelingresults difference between the configurations in two consec-utive iterations is lower than a predefined threshold. Thisstrategy can also help to avoid unnecessary time cost. Aftertermination of the AccD explorer, the ”best” configuration withmaximum design performance under the given constraints willbe output as the ”optimal” solution for the AccD design.

VII. EVALUATIONIn this section, we choose three representative benchmarks

(K-means, KNN-join, and N-body Simulation) and evaluatetheir corresponding AccD designs on the CPU-FPGA plat-form.

a) K-means: K-means [1, 18, 28–30] clusters a set ofpoints into several groups in an iterative manner. At eachiteration, it first computes the distances between each point andall clusters, and then update the clusters based on the averageposition of their inside points. We choose it as our benchmarksince it can show the benefits of AccD hierarchy (Trace-based+ Group-level) bound computation optimization on iterativealgorithms with disjoint source and target set.

b) KNN-join: KNN-join Search [2, 19, 31] finds the Top-K nearest neighbor points for each point in the source setfrom the target set. It first computes the distances betweeneach source point and all the target points. Then it ranksthe K-smallest distances for each source point and gets itscorresponding closest Top-K target points. KNN-join canhelp to demonstrate the effectiveness of AccD hybrid (Two-landmark + Group-level) bound computation optimizationon non-iterative algorithms.

c) N-body Simulation: N-body Simulation [32, 33] mim-ics the particle movement within a certain range of 3D space.At each time step, distances between each particle and itsneighbors (within a radius R) are first computed, and thenthe acceleration and the new position of each particle will beupdated based on these distances. While N-body simulationis also iterative, it has several differences compared with K-means algorithm: 1) N-body simulation has the same dataset(particles) for source and target set, whereas K-means operateson different source (point) and target (cluster) sets; 2) Allpoints in the N-body simulation would change their positionsaccording to the time variation, whereas in K-means only thetarget set (cluster) would change their positions during thecenter update; 3) N-body simulation has the same size ofsource and target set, whereas K-means target set (cluster)is much smaller than source set (point) in general. N-bodysimulation can help us to show the strength of AccD hybridbound computation (Two-landmark + Trace-based + Group-level) on iterative algorithms with the same source and targetset.A. Experiment Setup

a) Tools and Metrics: In our evaluation, we use IntelStratix 10 DE10-Pro [34] as the FPGA accelerator and runthe host side software program on Intel Xeon Silver 4110 pro-cessor [35] (8-core 16-thread, 2.1GHz base clock frequency,85W TDP). DE10-Pro FPGA has 378,000 Logic elements

(LEs), 128,160 adaptive logic modules (ALM), 512,640 ALMregisters, 648 DSPs, and 1,537 M20K memory blocks. Weimplement AccD design on DE10-Pro by using Intel QuartusPrime Software Suite [36] with Intel OpenCL SDK included.To measure the system power consumption (Watt) accurately,we use the off-the-shelf Poniie PN2000 as the external powermeter to get the runtime power of Xeon CPU and DE10 ProFPGA.

TABLE IV: Implementation Description.

Name Techniques DescriptionBaseline Standard Algorithm

without anyoptimization, CPU.

Naive for-loop basedimplementation onCPU.

TOP Point-basedTriangle-inequality

OptimizedAlgorithms, CPU.

TOP [11] optimizeddistance-relatedalgorithm running onCPU.

CBLAS CBLAS libraryAccelerated

Algorithms, CPU.

Standarddistance-relatedalgorithm withCBLAS [37]acceleration.

AccD Algorithmic-hardwareco-design,

CPU-FPGA platform.

GTI filtering andFPGA acceleration ofdistancecomputations.

b) Implementations: The CPU-based implementationsconsist of three types of programs: the naive for-loop se-quential implementation without any optimization (selected asour Baseline to normalize the speedup and energy-efficiency),the algorithm optimized by TOP [11] framework and thealgorithm optimized by CBLAS [37] computing library. Notethat the TOP + CBLAS implementation is not included in ourevaluation, since after applying TOP point-based TI filtering,each point in the source set has a distinctive list of pointsfrom the target set for distance computation, whereas CBLASrequires uniformity in the distance computations. Therefore, itis challenging to combine TOP and CBLAS optimization.

c) Dataset: In the evaluation, we use six datasets foreach algorithm. The selected datasets can cover the widespectrum of mainstream datasets, including datasets from UCIMachine Learning Repository [38], and datasets that haveever been used by previous papers [9, 11, 12] in the relateddomains. Details of these datasets are listed in Table V.Note that KNN-join algorithm will find the Top-1000 closestneighbors of each query point.

B. Comparison with Software Implementationa) Performance Comparison: As shown in Figure 8,

TOP, CBLAS, and AccD achieve average 9.12×, 9.19× and31.42× compared with Baseline across all algorithm anddataset settings, respectively. As we can see, AccD design canalways maintain the highest speedup among these implementa-tions. This largely dues to AccD GTI optimization in reducingdistance computation and its efficient hardware acceleration ofthe distance computation on FPGA.

We also observe that TOP implementation shows its strengthfor large datasets. For example, on dataset 3D Spatial Net-work (n = 434, 874) in KNN-join, TOP implementation

K-means KNN-join N-body SimulationDataset Size Dimension #Cluster Dataset Dimension #Source Dataset #ParticlePoker Hand 25,010 11 158 Harddrive1 64 68,411 P-1 16,384Smartwatch Sens 58,371 12 242 Kegg Net Directed 24 53,413 P-2 32,768Healthy Older People 75,128 9 274 3D Spatial Network 3 434,874 P-3 59,049KDD Cup 2004 285,409 74 534 KDD Cup 1998 56 95,413 P-4 78,125Kegg Net Undirected 65,554 28 256 Skin NonSkin 4 245,057 P-5 177,147Ipums 70,187 60 265 Protein 11 26,611 P-6 262,144

TABLE V: Datasets for Evaluation.

3.3

2.2

8

2.5

1

3.7

5

3.9

4

6.8

61.37

2.7

2.8

5

11

.78 6.5

4

9.4

7

39

.08

21

.95

21

.48

51

.61

23

.5

66

.61

0

10

20

30

40

50

60

70

Poker Hand SmartwatchSens

HealthyOlder People

KDD Cup2004

Kegg NetUndirected

Ipums

Spee

du

p (

x)

TOP CBLAS AccD

(a)

15

.17

12

.68

39

.78

8.3

7

26

.08

3.4

8

31

.77

12

.51

13

.22

27

.17

2.4

8

5.0

4

50

.57

27

.25

43

.17

88

.95

28

.21

21

.63

0

10

20

30

40

50

60

70

80

90

100

Harddrive1 Kegg NetDirected

3D SpatialNetwork

KDD Cup1998

Skin NonSkin Protein

Spee

du

p (

x)

TOP CBLAS AccD

(b)

4.6

8

5.2

3

5.7

8

6.0

9

6.9

1

7.3

0

2.1

5

7.4

6

7.2

2

7.2

5

7.1

5

7.2

4

9.1

4

13

.14

13

.19

14

.39

17

.46 1

4.2

0

2

4

6

8

10

12

14

16

18

20

P-1 P-2 P-3 P-4 P-5 P-6

Spee

du

p (

x)

TOP CBLAS AccD

(c)

Fig. 8: Performance Comparison (TOP, CBLAS, AccD): (a) K-means (b) KNN-Join (c) N-body Simulation. Note: Speedup isnormalized w.r.t Baseline.

3.3

2.2

8

2.5

1

3.7

5

3.9

4

6.8

6

0.6

3

1.2

5

1.3

8

4.5

3

2.9

5

3.7

5

16

2.3

89

.14 4

8.5

9

84

.41

65

.14

25

1.5

2

0

50

100

150

200

250

300

Poker Hand SmartwatchSens

Healthy OlderPeople

KDD Cup 2004 Kegg NetUndirected

Ipums

Ener

gy E

ffic

ien

cy (x

)

TOP CBLAS AccD

(a)

15

.17

12

.68

39

.78

8.3

7

26

.08 3

.48

9.3

8

3.9

3

3.9

5

7.7

3

0.8

1

1.6

14

7.1

9

10

0.2

7

12

3.5

7

21

9.1

1

92

.62

76

.81

0

50

100

150

200

250

Harddrive1 Kegg NetDirected

3D SpatialNetwork

KDD Cup 1998 Skin NonSkin Protein

Ener

gy E

ffic

ien

cy (x

)

TOP CBLAS AccD

(b)

4.6

8

5.2

3

5.7

8

6.0

9

6.9

1

7.31.2

4

3.6

3

2.7

7

2.6

2.6

4

2.0

3

37

.47

49

.73

64

.37 61

.53

73

.77

45

.87

0

10

20

30

40

50

60

70

80

P-1 P-2 P-3 P-4 P-5 P-6En

ergy

Eff

icie

ncy

(x)

TOP CBLAS AccD

(c)

Fig. 9: Energy Efficiency Comparison (TOP, CBLAS, AccD): (a) K-means. (b) KNN-Join. (c) N-body Simulation. Note: EnergyEfficiency is normalized w.r.t Baseline.

achieves 39.78× speedup. Since the fine-grained point-basedTI optimization of TOP can reduce most (more than 90%)of the unnecessary distance computations, which benefits theoverall performance to a great extent. Note that the intrinsicpoint distribution of the dataset would also affect the filteringperformance of TOP, but in general, the larger dataset couldlead TOP to spot and remove more redundant computations.

What we also notice is that CBLAS implementation demon-strates its performance on datasets with relatively high dimen-sionality. For example, on dataset KDD Cup 2004 (d = 74)in the K-means algorithm, CBLAS achieves 11.78× speedupover Baseline, which is higher than its performance on otherK-means datasets. This is because, on high dimension dataset,CBLAS implementation can get more benefits of parallelizedcomputing and more regularized memory access, whereas, inlow dimension settings, the same optimization can only yieldminor speedup.

Our AccD design achieves a considerable speedup ondatasets with large size and high dimensionality. For example,on dataset KDD Cup 2004 (n = 285, 409, d = 74) and Ipums(n = 70, 187, d = 60) in K-means, AccD achieves 51.61× and

66.61× speedup over Baseline, and also significantly higherthan both TOP and CBLAS implementations. This conclusioncan also be extended to KNN-join, such as 88.95× speedupon dataset KDD Cup 1998 (n = 95, 413, d = 56). Sinceour AccD design can effectively reconcile the benefits fromboth the GTI optimization and the FPGA acceleration, wherethe former provides the opportunity to reduce the distancecomputation at the algorithm level, and the latter boosts theperformance from hardware acceleration perspective. Moreimportantly, our AccD design can balance the above twobenefits to maximize the overall performance.

b) Energy Comparison: The energy efficiency of AccDdesign is also significant. For example, on the K-meansalgorithm, AccD designs deliver an average 116.85× better en-ergy efficiency compared with Baseline, which is significantlyhigher than TOP and CBLAS implementations. There arenamely two reasons behind these results: 1) Much lower powerconsumption. AccD CPU-FPGA design only consumes 5w ∼17.12w across all algorithm and dataset settings, whereas IntelXeon CPU consumes at least 20.9w and 42.49w on TOPand CBLAS implementations, respectively; 2) Considerable

performance. AccD design achieves a much better speedup(more than 5× on average) compared with the TOP andCBLAS, which contributes to overall design energy-efficiency.

Among these implementations, CLBAS implementation hasthe lowest energy efficiency, since it relies on multi-coreparallel processing capability of the CPU, which improves theperformance at the cost of much higher power consumption(average 65.79w). TOP only leverages the single-core process-ing capability of the CPU and achieves moderate performancewith effective distance computation reduction, which results inless power consumption (average 25.59w) and higher energyefficiency (average 9.12×) compared with Baseline. Differentfrom the TOP and CBLAS implementations, AccD design isbuilt upon a low-power platform with considerable perfor-mance, which shows a far better energy-performance trade-off.

C. Performance Benefits AnalysisTo analyze the performance benefits of AccD CPU-FPGA

design in detail, we use K-means as the example algorithm forstudy. Specifically, we build four implementations for compar-ison: 1) TOP K-means on CPU; 2) TOP K-means on CPU-FPGA platform; 3) AccD K-means on CPU; 4) AccD K-meanson CPU-FPGA platform. Note that TOP K-means is designedfor sequential-based CPUs, and no publicly available TOP im-plementation on CPU-FPGA platforms. For a fair comparison,we implement TOP K-means on CPU-FPGA platform withmemory optimizations (inter-group and intra-group memoryoptimization) and distance computation kernel optimization(Vector-Matrix multiplication). These optimizations improvethe data reuse and memory access performance. We compute

3.3

0

2.2

8

2.5

1

3.7

5

3.9

4

6.8

6

3.5

3

2.1

2

2.3

8

4.4

2

1.1

5

2.2

6

3.0

3

1.7

7

2.2

2

3.3

1

1.6

2

4.2

4

39

.08

21

.95

21

.48

51

.61

23

.50

66

.61

0

10

20

30

40

50

60

70

Harddrive1 Kegg NetDirected

3D SpatialNetwork

KDD Cup 1998 Skin NonSkin Protein

Spe

ed

up

(x)

TOP (CPU) TOP (CPU-FPGA)AccD (CPU) AccD (CPU-FPGA)

Fig. 10: AccD Performance Benefits Breakdown.

the normalized speedup performance of each implementationw.r.t the naive for-loop based K-means implementation onCPU.

As shown in Figure 10, AccD K-means on CPU-FPGA plat-form can always deliver the best overall speedup performanceamong these implementations. We also observe that TOP K-means can achieve average 3.77× speedup on CPU, however,directly porting this optimization towards CPU-FPGA plat-form could even lead to inferior performance (average 2.63×).Even though we manage to add several possible optimizations,applying such fine-grained TI optimization from TOP wouldstill cause a large divergence of computation among points,leading to low data reuse and inefficient memory access.

We also notice that AccD design on CPU achieves lowerspeedup (average 2.69×) compared with the TOP (average

3.77×), since its coarse-grained GTI optimization spots afewer number of unnecessary distance computations. However,when combining AccD design with CPU-FPGA platform,the benefits of AccD GTI optimization become prominent(average 37.37×), since it can maintain computation regularitywhile reducing memory overhead to facilitate the hardware ac-celeration on FPGA. Whereas, applying optimization to max-imize the algorithm-level benefits while ignoring hardware-level properties would result in poor performance, such as theTOP (CPU-FPGA) implementation. Moreover, comparison ofAccD (CPU) and AccD (CPU-FPGA) can also demonstratethe effectiveness of using FPGA as the hardware acceleratorto boost the performance of the algorithms, which can deliveradditional 9.68× ∼ 15.71× speedup compared with thesoftware-only solution.

VIII. CONCLUSIONIn this paper, we present our AccD compiler framework

to accelerate the distance-related algorithms on the CPU-FPGA platform. Specifically, AccD leverages a simple butexpressive language construct (DDSL) to unify the distance-related algorithms, and an optimizing compiler to improve thedesign performance from algorithmic and hardware perspec-tive systematically and automatically. Rigorous experiments onthree popular algorithms (K-means, KNN-join, and N-bodysimulation) demonstrate the AccD as a powerful and com-prehensive framework for hardware acceleration of distance-related algorithms on the modern CPU-FPGA platforms.

REFERENCES[1] S. Lloyd. Least squares quantization in pcm. IEEE Transactions on

Information Theory, 1982.[2] Naomi S Altman. An introduction to kernel and nearest-neighbor

nonparametric regression. The American Statistician, 1992.[3] M. Trenti and P. Hut. N-body simulations (gravitational). Scholarpedia,

2008.[4] H. M. Hussain, K. Benkrid, H. Seker, and A. T. Erdogan. Fpga

implementation of k-means algorithm for bioinformatics application: Anaccelerated approach to clustering microarray data. In 2011 NASA/ESAConference on Adaptive Hardware and Systems (AHS), 2011.

[5] Zhongduo Lin, Charles Lo, and Paul Chow. K-means implementationon fpga for high-dimensional data using triangle inequality. In 22nd In-ternational Conference on Field Programmable Logic and Applications(FPL), 2012.

[6] T. Saegusa and T. Maruyama. An fpga implementation of k-meansclustering for color images based on kd-tree. In 2006 InternationalConference on Field Programmable Logic and Applications (FPL),2006.

[7] Zhe-Hao Li, Ji-Fang Jin, Xue-Gong Zhou, and Zhi-Hua Feng. K-nearestneighbor algorithm implementation on fpga using high level synthesis. In2016 13th IEEE International Conference on Solid-State and IntegratedCircuit Technology (ICSICT), 2016.

[8] Charles Elkan. Using the triangle inequality to accelerate k-means. InProceedings of the 20th International Conference on Machine Learning(ICML), 2003.

[9] Yufei Ding, Yue Zhao, Xipeng Shen, Madanlal Musuvathi, and ToddMytkowicz. Yinyang k-means: A drop-in replacement of the classic k-means with consistent speedup. In International Conference on MachineLearning (ICML), 2015.

[10] J. Canilho, M. Vstias, and H. Neto. Multi-core for k-means clusteringon fpga. In 2016 26th International Conference on Field ProgrammableLogic and Applications (FPL), 2016.

[11] Yufei Ding, Xipeng Shen, Madanlal Musuvathi, and Todd Mytkowicz.Top: A framework for enabling algorithmic optimizations for distance-related problems. Proc. VLDB Endow., 2015.

[12] Guoyang Chen, Yufei Ding, and Xipeng Shen. Sweet knn: An efficientknn on gpu through reconciliation between redundancy removal andregularity. In 2017 IEEE 33rd International Conference on DataEngineering (ICDE), 2017.

[13] Ioannis Stamoulias and Elias S. Manolakos. Parallel architectures forthe knn classifier – design of soft ip cores and fpga implementations.ACM Trans. Embed. Comput. Syst., 2013.

[14] E. S. Manolakos and I. Stamoulias. Ip-cores design for the knn classifier.In Proceedings of 2010 IEEE International Symposium on Circuits andSystems, 2010.

[15] H. M. Hussain, K. Benkrid, A. T. Erdogan, and H. Seker. Highlyparameterized k-means clustering on fpgas: Comparative results withgpps and gpus. In 2011 International Conference on ReconfigurableComputing and FPGAs, 2011.

[16] Dominique Lavenier. Fpga implementation of the k-means clusteringalgorithm for hyperspectral images. Los Alamos National LaboratoryLAUR, 2000.

[17] G. Di Fatta and D. Pettinger. Dynamic load balancing in parallel kd-treek-means. In 2010 10th IEEE International Conference on Computer andInformation Technology (ICCIT), 2010.

[18] Tapas Kanungo, David M Mount, Nathan S Netanyahu, Christine DPiatko, Ruth Silverman, and Angela Y Wu. An efficient k-meansclustering algorithm: Analysis and implementation. IEEE Transactionson Pattern Analysis & Machine Intelligence (PAMI), 2002.

[19] C. Yang, X. Yu, and Y. Liu. Towards efficient knn joins on data streams.In 2014 IEEE International Congress on Big Data, 2014.

[20] P. Sirait and A. M. Arymurthy. Cluster centres determination basedon kd tree in k-means clustering for area change detection. In 2010International Conference on Distributed Frameworks for MultimediaApplications, 2010.

[21] Ruicheng Zhong, Guoliang Li, Kian-Lee Tan, and Lizhu Zhou. G-tree:An efficient index for knn search on road networks. In Proceedings ofthe 22nd ACM International Conference on Information & KnowledgeManagement (CIKM), 2013.

[22] Y. Wang, Z. Zeng, B. Feng, L. Deng, and Y. Ding. Kpynq: A work-efficient triangle-inequality based k-means on fpga. In 2019 IEEE27th Annual International Symposium on Field-Programmable CustomComputing Machines (FCCM), 2019.

[23] Xing Zhang Zhiqiang Yang Jipan Huang Miren Tian, Xin’an Wang andHao Chen. The implementation of a knn classifier on fpga with a paralleland pipelined architecture based on predetermined range search. In2016 13th IEEE International Conference on Solid-State and IntegratedCircuit Technology (ICSICT), 2016.

[24] H. Hussain, K. Benkrid, C. Hong, and H. Seker. An adaptive fpgaimplementation of multi-core k-nearest neighbour ensemble classifierusing dynamic partial reconfiguration. In 22nd International Conferenceon Field Programmable Logic and Applications (FPL), 2012.

[25] H. M. Hussain, K. Benkrid, and H. Seker. An adaptive implementationof a dynamically reconfigurable k-nearest neighbour classifier on fpga.In 2012 NASA/ESA Conference on Adaptive Hardware and Systems(AHS), 2012.

[26] Greg Hamerly. Making k-means even faster. In Proceedings of the 2010SIAM international conference on data mining (SIAM), 2010.

[27] H. Hong, G. Juan, and W. Ben. An improved knn algorithm based onadaptive cluster distance bounding for high dimensional indexing. In2012 Third Global Congress on Intelligent Systems, 2012.

[28] Anil K Jain. Data clustering: 50 years beyond k-means. Patternrecognition letters, 2010.

[29] Adam Coates and Andrew Y Ng. Learning feature representations withk-means. In Neural networks: Tricks of the trade. 2012.

[30] Siddheswar Ray and Rose H Turi. Determination of number of clustersin k-means clustering and application in colour image segmentation. InProceedings of the 4th international conference on advances in patternrecognition and digital techniques, 1999.

[31] Michael Gowanlock. Knn-joins using a hybrid approach: Exploitingcpu/gpu workload characteristics. In Proceedings of the 12th Workshopon General Purpose Processing Using GPUs (GPGPU), 2019.

[32] Lars Nylons. Fast n-body simulation with cuda. 2007.[33] Shigeru Ida and Junichiro Makino. N-body simulation of gravitational

interaction between planetesimals and a protoplanet: I. velocity distri-bution of planetesimals. 1992.

[34] Terasic de10-pro stratix 10 gx/sx fpga development kit. URL http://www.terasic.com.tw/cgi-bin/page/archive.pl?CategoryNo=248&No=1144.

[35] Intel xeon silver 4110 processor. URL https://www.intel.com/content/www/us/en/products/processors/xeon/scalable/silver-processors/silver-4110.html.

[36] Intel quartus prime software suite. https://www.intel.com/content/www/us/en/software/programmable/quartus-prime/overview.html.

[37] Qian Wang, Xianyi Zhang, Yunquan Zhang, and Qing Yi. Augem:automatically generate high performance dense linear algebra kernelson x86 cpus. In Proceedings of the International Conference on HighPerformance Computing, Networking, Storage and Analysis (SC), 2013.

[38] Dheeru Dua and Casey Graff. UCI machine learning repository, 2017.URL http://archive.ics.uci.edu/ml.

http://www.terasic.com.tw/cgi-bin/page/archive.pl?CategoryNo=248&No=1144http://www.terasic.com.tw/cgi-bin/page/archive.pl?CategoryNo=248&No=1144https://www.intel.com/content/www/us/en/products/processors/xeon/scalable/silver-processors/silver-4110.htmlhttps://www.intel.com/content/www/us/en/products/processors/xeon/scalable/silver-processors/silver-4110.htmlhttps://www.intel.com/content/www/us/en/products/processors/xeon/scalable/silver-processors/silver-4110.htmlhttps://www.intel.com/content/www/us/en/software/programmable/quartus-prime/overview.htmlhttps://www.intel.com/content/www/us/en/software/programmable/quartus-prime/overview.htmlhttp://archive.ics.uci.edu/ml

I IntroductionII Related WorkII-A Algorithmic OptimizationII-B Hardware Acceleration

III Distance-related Algorithm Domain-Specific Language (DDSL)III-A Data ConstructIII-B Distance Compute ConstructIII-C Distance Selection ConstructIII-D Data Update ConstructIII-E Iteration ConstructIII-F Example: K-means

IV Algorithm OptimizationIV-A TI in Distance-related AlgorithmIV-B Generalized Triangle Inequality (GTI)

V Hardware AccelerationV-A Memory OptimizationV-B Distance Computation Kernel

VI AccD CompilerVI-A Design Parameters and ConstraintsVI-B Design Space Exploration

VII EvaluationVII-A Experiment SetupVII-B Comparison with Software ImplementationVII-C Performance Benefits Analysis

VIII Conclusion