1

Beam Detection Based onMachine Learning Algorithms

Haoyuan Li, hyli16@stanofrd.edu Qing Yin, qingyin@stanford.edu

Abstract—The positions of free electron laser beams onscreens are precisely determined by a sequence of machinelearning models. Transfer training is conducted in a self-constructed convolutional neural network based on VGG16model. Output of intermediate layers are passed as featuresto a support vector regression model. With this sequence,85.8% correct prediction is achieved on test data.

Index Terms—Beam Detection, SVM, CNN

I. INTRODUCTION

THE free electron laser(FEL) at Stanford LinearAccelerator Center(SLAC) is an ultra-fast X-ray

laser. As one of the most advanced X-ray light source[1] [2], it is famous for its high brightness and shortpulse duration: it is 10 billion times brighter than theworld’s second brightest light source; the pulse durationis several tens femtoseconds.It plays a pivotal role inboth fundamental science research and applied research[2]. The mechanism behind this laser is very delicate[1]. Thus to keep the laser in optimal working conditionis challenging.The positions of the electron beams andthe laser beams are of fundamental importance in thecontrol and maintenance of this FEL.

Currently, the task of locating beam spots heavilydepends on human labor. This is mainly attributed tothe wide varieties of beam spots and the presentationof strong noises as demonstrated in Figure 1, wherethe white square marks the boundary of the beam spot.Each picture requires a long sequence of signal pro-cessing methods to mark the beam position. To makethings even worse, different instrument configurationsand working conditions require different processing pa-rameters. Within the frame of same configuration, theparameters will also drift away along with time advance-ment because of the inherent delicacy of the instrument.This makes tuning and maintaining the FEL a tediousand burdensome task for researchers.Currently, the dataupdate frequency of the laser is 120 Hz. We can barelyhandle this. In 2020, after the scheduled update, thefrequency will climb to 5000 Hz. Thus the only hopelies in automatic processing methods. Considering thatsimple signal processing method can not handle such

Fig. 1: Raw photos

nasty condition, We hope to come up with a generalmachine learning algorithm which is capable of locatingthe beams’ positions quickly and automatically. In thisreport, we demonstrate the consecutive application ofneural network and supportive vector machine (SVM) todetermine the positions of beam spots on virtual cathodecamera (VCC) screen in simple cases.

II. CHALLENGES AND STRATEGIES

A bird view of such a simple-looking task reveals thechallenges behind it.

• Complicated background noises. The backgroundnoises are not static. In contrast, it is coherentin both time and space domain due to quantum

2

effect. Thus we have to dynamically change thebackground when doing background subtraction fordifferent images.

• Large variety in the intensity and shape of the beamspot. As is indicated above, the maximum intensityof the beam is incredibly high. However, it can alsobe 0 which corresponds to the case that no beamspot presented. The shape of the beam can alsovaries significantly from case to case.

• Ground truths is hard to find. The signal processingmethods fail in finding the beam spot sometimes.There are also cases when the photo is so terriblethat even human can not find the beam spot afterall the signal processing procedures.

The task is similar to face recognition and the ex-pectation of solving it in one strike is unrealistic. Thus,we develop the following strategy. First, we develop analgorithm that is capable of dealing with cases wherewe have implemented signal processing to realize thefundamental functions. Second, extend the applicationto cases that we gradually ignore pre-processing steps,e.g. Gaussian blur. In the end, extend the application toraw images. In the essay, we report the accomplishmentof the first step that realizes fundamental functions.

III. RELATED WORK

A similar dilemma stated above occurred when re-searchers tried to pinpoint the electron bunch’s trace[3]. It was solved by first feeding the images intoa convolutional neural network [4], VGG16 [5], andthen performing linear regression on its codewords. Thisgives us the inspiration of this project. Signal processingalgorithms have been developed for targeting the beamspot. Thus, we use the results from these algorithms asthe ground truth for our training.

IV. DATASET

Training dataset for this project consists of 162 origi-nal VCC screen figures and 16200 figures generated fromthe original ones (each original image corresponding to100 generated images). Each figure contains only onebeam spot. To generate new figures, we first cut out partof figure that contains the beam spot, then rotate thesmall patch of figure to some random degree, and putthe patch on a random point. In the end, cut out thecovered part of the figure and paste it to the originalposition of the beam spot. The position of the beamon each of the original figures can be determined bythe signal processing algorithm. The position of thebeam on the generated figures are determined by thegeneration process.The position of beam in each figure is

1620016200

Fig. 2: Algorithmic flowchart

represented by the coordinates of two diagonal vertices:z(i) = (y

(i)min, y

(i)max, x

(i)min, x

(i)max)T

V. METHOD

A. Pipeline

To obtain an algorithm robust to variations of beamspot and noises, we utilize a two step method.

First, we use a convolutional neural network (CNN) asa feature extractor and we take the intermediate outputsof the CNN as features and train a Supportive VectorRegression(SVR) program with them. The pipeline isdemonstrated in Figure 2.

B. Feature Extraction

Since we have moderate sample size, transfer trainingseems suitable for this purpose. VGG16 is selected asthe pre-trained model.

3

Fig. 3: Structure of VGG 16 [6]

VGG16 is a deep convolutional neural network. Thereare totally 13 layers of convolutional layer, 5 poolinglayer and 3 full connected layer. The structure of thisnetwork in demonstrated in Figure 3.

The output of the first and second fully connectedlayers are both 4096-dimensional vectors. We feed inour samples and use these two vectors as the featuresof the image.

To do transfer training, first we re-construct theVGG16 with python 2.7 and tensorflow 0.11. Thelast layer of the original VGG16 is a classifier of athousand classes. We modify it to output the positionof the center of the beam spot. The loss is defined asthe squared distance between the predicted beam centerand the true beam center. A l2 regularization term isadded which contains parameters of the last three fullyconnected layer.

LCNN,loss =1

m

m∑i=1

||~xii− ~xi||2 +λ

m||W ||2 (1)

where ~xi and ~xi indicate respectively the predictedcenter of the beam and the true center of the beamwhich is represented by the center of the box. W is avector consisting of all parameters in the last three fullconnected layers, i.e. we only tune the parameters inthe last three layers.

The reason for only tuning the last three layers isthat we don’t have enough data to tune all parameters.The first several layers, according to modern viewpoint,mainly detect the detailed structures while the upper lay-ers concern more about larger scale objects. So it seemsmore reasonable to try to tune those layers affecting ournext step directly.

C. Support Vector Regression

The feature for the ith figure is denoted by q(i) ∈R8192.To avoid overfitting, we use l2 regularization whenimplementing supportive vector regression (SVR) withGaussian kernel to model the situation. The four param-eters of the position are independent. So we train a SVRmodel for each of the four labels by maximizing thefollowing dual function [7]

Ls =m∑i=1

αsi z

(i)s −

1

2

m∑i,j=1

αsiα

sjK(q(i), q(j)) (2)

αi ∈ [−t, t], i ∈ {1, 2, · · · ,m}, s = 1, 2, 3, 4. m is thenumber of training samples. αs

i ’s are the optimization pa-rameters. t is the penalty ratio of the difference betweenthe predicted label and the real label. This dual functionis obtained from the primary optimization problem [7]in the following equations by eliminating the originalvariables:

Ls =1

2|θs|22 + t

m∑i=1

ξ+i + t

m∑i=1

ξ−i (3)

under the constraints that ∀i ∈ {1, 2, · · · ,m}

ξ+i ≥ 0 (4)

ξ−i ≥ 0 (5)

z(i)s − θT q(i)s ≤ ξ+i (6)

θT q(i)s − z(i)s ≤ ξ−i (7)

where ξ+i and ξ−i are the threshold for training error.

VI. TRAINING

A. Convolutional Neural Network

Nesterov momentum method [8] is utilized to mini-mize the CNN loss function. We use a vanishing learningrate which decays as t−1:

α = 10−4 × (1 + t)−1 (8)

B. Cross Validation

To maximize the usage of limited data, we use 9-fold cross-validation. First we randomly shuffle the 162original images to erase the potential correlation betweendifferent images. Then divided them into 9 batches, eachof 18 images. For each batch, the other 144 originalimages together with the related 14400 generated imagesform the training data set for the next two steps. Thisguarantees the independence between the training andtesting process.

4

C. PCA and SVM

Sequential minimal optimization (SMO) algorithm isutilized to minimize the SVR loss function. Packagescikit-learn is used throughout the process to guaranteethe performance.

The feature has a huge dimension of 8192, whilethe prediction only contains 4 numbers. To preventover-fitting, we use principle component analysis (PCA)to extract the most influence dimensions and iteratethe training and testing procedure for 12 differentdifferent dimensions equally distributed in log-scale, i.e.{5, 10, 20, 40, 80, 160, 320, 640, 1280, 2560, 5120, 8000}.

VII. RESULTS AND DISCUSSIONS

A. Performance

After re-tuning, the regression algorithm givessuperior performance. Beam positions are correctedpredicted in 139 of the 162 original images. Thefollowing diagram demonstrates the typical performanceof the regression on the test set. When the regressionalgorithm catches the spot, it gives really preciseprediction, which implies that the algorithm does havelearned how to pinpoint the position of the spot.

Yet, when it can not catch the spot (picture in secondcolumn of Figure 4), the regression algorithm just ran-domly guess a point in the center of the screen. Afterchecking the pictures where the algorithm miss the spot,we found that the missing spots are not dim at all. In factthe maximum point in those figures are even larger thanthose with perfection predictions. But the beam spots areall very small in those figures.

B. Effect of PCA dimension

To prevent overfitting, we use PCA to pick out themost influential dimensions of the features before regres-sion. Figure 5 demonstrates the training error of the SVRfor different PCA dimensions. The error is defined asthe distance between the predicted beam center and thetrue beam center. Figure 6 demonstrate the performanceof the SVR for different PCA dimensions.To quantifythe performance, we use the average ratio between theoverlapped area and the area of the true spot as theindicator of the performance.

From these two figures, it is clearly shown that whenthe PCA dimension exceeds 1000, there will be severeoverfitting problem, as the training vanishes and theperformance drops in that area.

Fig. 4: Test Results

PCA Dimension10

010

110

210

310

4

Overlap

are

ara

tio

0.6

0.62

0.64

0.66

0.68

0.7

0.72

0.74

Fig. 5: Test performance for different PCA dimensions

VIII. CONCLUSIONS AND FUTURE WORKS

With newly generated figures and re-tuned neuralnetwork, the performance of the SVR gains great im-provement. Yet it is still far from applicable in realenvironment. Three approaches are scheduled for furtherimprovements.

A. Fine-Tuning Neural Network

VGG16-like neural networks are huge and hard totune. It is highly likely that with improved tuning skilland a bit of luck and patience, we can get betterperformance.

5

0 1000 2000 3000 4000 5000 6000 7000 8000Sample label

0

2

4

6

8

10

Center Deviation

d=5d=160d=1280

Fig. 6: Training error for different PCA dimensions

Fig. 7: Test results for raw pictures

B. More and Harder Data

Noisy figures are currently beyond the signal process-ing algorithm’s capability. But as we can see sometimesour regression algorithm has done better job on theexisting samples. And even for those raw pictures, ourmodel can generate good prediction results occasionally(Figure 7). So we may try the algorithm on hardersamples in the future. Especially we will try to markthe position of the beam by hand for noisy figures andtry to train the model on those samples as well.

C. Building New Neural Network

It is possible that a smaller but better training neuralnetwork can do better job. Only after trying both methodscan we decide which way is better. We have alreadyconstructed another smaller neural network. Training ofthis one will soon begin.

REFERENCES

[1] Z. H. Kwang-Je Kim and R. Lindberg, Synchrotron Radiationand Free-Electron Lasers. Cambridge University Press, 2017.

[2] C. Pellegrini, A. Marinelli, and S. Reiche, “Thephysics of x-ray free-electron lasers,” Rev. Mod. Phys.,vol. 88, p. 015006, Mar 2016. [Online]. Available:http://link.aps.org/doi/10.1103/RevModPhys.88.015006

[3] “Spatial Localization - AI at SLAC- SLAC Confluence.” [Online]. Available:https://confluence.slac.stanford.edu/display/AI/Spatial+Localization

[4] “CS231n Convolutional Neural Networks for VisualRecognition.” [Online]. Available: http://cs231n.github.io/

[5] K. Simonyan and A. Zisserman, “Very Deep ConvolutionalNetworks for Large-Scale Image Recognition,” sep 2014.[Online]. Available: http://arxiv.org/abs/1409.1556

[6] “VGG in TensorFlow · Davi Frossard.” [Online]. Available:https://www.cs.toronto.edu/ frossard/post/vgg16/

[7] A. J. Smola and B. Scholkopf, “A tutorial on support vectorregression,” Statistics and computing, vol. 14, no. 3, pp. 199–222, 2004.

[8] Y. Bengio, N. Boulanger-Lewandowski, and R. Pascanu,“Advances in Optimizing Recurrent Networks,” dec 2012.[Online]. Available: http://arxiv.org/abs/1212.0901