Natural Language Processing Models That Automate Programming Will TransformChemistry Research and Teaching

Glen M. HockyDepartment of Chemistry, New York University∗

Andrew D. WhiteDepartment of Chemical Engineering, University of Rochester†

(Dated: August 31, 2021)

Natural language processing models have emerged that can generate usable software and automatea number of programming tasks with high fidelity. These tools have yet to have an impact on thechemistry community. Yet, our initial testing demonstrates that this form of Artificial Intelligenceis poised to transform chemistry and chemical engineering research. Here, we review developmentsthat brought us to this point, examine applications in chemistry, and give our perspective on howthis may fundamentally alter research and teaching.

In 2021, Chen et al. released a new natural languageprocessing (NLP) model called Codex that can generatecode from natural language prompts [1]. Interest hasbeen broadly focused on its application to software engi-neering. We, somewhat sarcastically, asked it to “Com-pute the dissociation curve of H2 using pyscf” [2] and theresult is shown in Fig. 1. It generated correct code andeven plotted it (see SI for further details). Some mayscoff at the artificial intelligence (AI) selected method(Hartree–Fock) and basis set (STO-3G). Thus, we askedit to “use the most accurate method” as a continuationof our “conversation” and it computed the dissociationcurve using CCSD(T) in a large basis. AI models thatcan connect natural language to programming will havesignificant consequences to the field of chemistry—herewe outline a brief history of these models and our per-spective on where these models will take us.

Recent developments

There has been a flurry of advances in the topic of“autocomplete” style language models that can generatetext given a prompt over the last three years. These lan-guage models are deep neural networks with a specificarchitecture called transformers [3, 4]. These models aretrained on text that has words hidden [5], and have thetask of filling in missing text [4, 6, 7]. This is called “pre-training,” because these models were not intended to fillin missing words, but rather be used on downstream taskslike classifying sentiment in text or categorizing text [4].Surprisingly, it was found that these models could gen-erate a long seemingly real passage of text simply froma short initial fragment of text called a prompt [4, 8].These prompts can be to answer a question, summarizea story, or make an analogy—all with the same model.This was interesting, especially because the quality wasbeyond previous text generation methods like recurrentneural networks or hidden Markov models [9]. After in-creasing model size and the training corpus, the next

0.5 1.0 1.5 2.0 2.5 3.0Length of bond (Angstrom)

1.1

1.0

0.9

0.8

0.7

Tota

l ene

rgy

(Har

tree)

FIG. 1. Prompt: Compute the dissociation curve of H2

using the pyscf library. See SI for code. Note, whenrepeating this prompt, the method, labels, and range canchange due to under-specification of the request.

generation of language models were able to answer novelprompts beyond standard question-and-answer or writ-ing summaries [10]. For example, given three worked outexamples of extracting compound names from a sentence,the GPT-3 model could do the same for any new sen-tence. We show the utility of this for parsing chemistryliterature in Fig. 2 using text from Ref. [11] (see SI forfull details). This result is remarkable because it requiresno additional training, just the input prompt – literally atraining size of 3. Not so long ago, this was considered adifficult problem even when using thousands of trainingexamples [12]. A caveat to these large language models isthat they have a limited understanding of the text whichthey parse or generate; for example, we find they can gen-erate seemingly valid chemistry text but cannot answersimple questions about well known chemical trends.

After these new large language models were developed,anyone could have state-of-the art performance on lan-guage tasks simply by constructing a few examples of

arX

iv:2

108.

1336

0v1

[co

nd-m

at.s

tat-

mec

h] 3

0 A

ug 2

021

2

Sentence: In brief, fluorescent dyes dissolved

in tetrahydrofuran (THF) were added to particles

stabilized with Pluronic F108 (MilliporeSigma)

to a final concentration of 30% v/v THF, then

diluted by a factor of five before washing the

particles via multiple sedimentation and

resuspension cycles to set them in pure water.

Chemical Entities: Pluronic F108,tetrahydrofuran, THF

FIG. 2. Example of chemical entity recognition after trainingon three examples with GPT-3. Prompt including a directquote of text from Ref. [11] is in monospace and response isbolded and red. Note that this misses the connection betweentetrahydrofuran and THF, and does not associate water witha chemical entity.

their task. In the last few months, even the need forworked out examples can be removed. In some cases asimple ‘imperative’ sentence is enough [13]. For example,a variation on the name of this article was generated byasking an imperative-style model to “write an excitingtitle” given an earlier version of the abstract. The pacehas been nothing short of remarkable, going from thetransformer in 2017 to a near universal language modelin 2020 to a model which can take instructions in 2021.

The largest and arguably most accurate model in thisclass is still the GPT-3 model from OpenAI [10]. GPT-3 is an enigma in the field of natural language models.It is democratizing because anyone can create a power-ful language model in a few hundred characters that isdeployable immediately. Yet its weights are a pseudo-trade secret, owned and licensed by OpenAI exclusivelyto Microsoft. Thus the only way to run it is via their web-site (or API). These kinds of models are known as LargeLanguage Models. Any state-of-the-art language modelsshould start with a large language model like GPT-3 or,for example, the freely available GPT-NEO [14]. GPT-3has been trained on billions of tokens and no effort hasyet to match its scale of training data and model size. Itcan be unsettling too because it has quite adeptly cap-tured the racism, sexism, and bias in human writing andcan be reflected in its responses [15]. Mitigating this isan ongoing effort [16]. Another interesting outcome isthat “prompt engineering,” literally learning to interfacemore clearly with an AI, is now a research topic [17].

GPT-3 has yet to make a major impact on chemistry,likely because it was available starting only in 2021. Wepreviously prepared a demo of voice-controlled moleculardynamics analysis using GPT-3 to convert natural lan-guage into commands [18]. Although an impressive ex-ample of voice controlled computational chemistry hadbeen published using Amazon’s Alexa [19], we found inour work that GPT-3 could handle looser prompts such“wait, actually change that to be ribbons.” It also took

only about a dozen examples to teach GPT-3 how todo tasks like render a protein, change its representation,and select specific atoms using VMD’s syntax [20]. Thisis a significant reduction in researcher effort to make suchtools, only taking a few hours total between the two ofus. Our program itself adds an element of accessibilityfor those who may have difficulty with a keyboard andmouse interface through this voice-controlled interface,and we could easily, and plan to, generalize this approachto other analysis software used in our groups.

Perhaps because programmers were the most excitedabout GPT-3, frequent usage examples involved the gen-eration of code. And thus we reach the present, withOpenAI’s release in August of a GPT-3 model tuned ex-plicitly for this purpose, termed Codex [1]. Althoughautomatic code generation in chemistry is not new (e.g.[21, 22]), we believe that the scope and natural languageaspects mean that Codex will have a broad impact onboth the computational and experimental chemistry com-munity.

Over time, there has been a tremendous increase in thenumber of available software packages to perform com-putational chemistry tasks. These off-the-shelf tools canenable students to perform tasks in minutes which mighthave taken a large portion of their Ph.D. to complete justten years ago. Yet now, a large fraction of a researcher’stime that used to be spent on repetitive coding taskshas been replaced by learning the interfaces to these nu-merous software packages; this task is currently done bya combination of searching documentation pages on theweb, reading and following tutorial articles, or simply bytrial and error. These new NLP models are able to elim-inate intermediate steps and allow researchers to get onwith their most important task, which is research! Somesuccessful examples we have tried are shown in Fig. 3,with full details in the SI.

Immediate impact on research and education

Scientific software. Many scientific programmingtasks, whether for data generation or data analysis, aretedious and often repetitive over the course of a long re-search project. Codex can successfully complete a widerange of useful scientific programming tasks in secondswith natural language instructions, greatly reducing timeto completion of many common tasks. These could in-clude writing a function to convert between two differentfile formats, producing well formatted plots with prop-erly labeled axes, converting LATEX equations into a func-tion, implementing standard algorithms such histogram-ming, adding comments to code, and converting codefrom one programming language to another [1]. We haveeven found that Codex is capable of performing someof these tasks using non-English prompts, which couldhelp reduce barriers to accessing software libraries facedby non-native speakers—although result accuracy whenusing non-English prompts has not been fully explored.

3

A 1. Load the caffeine molecule from SMILES2. Draw it

- This script generates the behavior of a 2D random walk and plots trajectories- Rewrite this into a function

B def gauss(smiles): """This function prepares a Gaussian input file from the given smiles using rdkit to prepare initial coordintes by 1. Converting SMILES to molecule 2. Preparing 3D coordinates 3. Writing out Gaussian input file""" # Convert SMILES to molecule mol = Chem.MolFromSmiles(smiles) # Generate 3D coordinates mol = Chem.AddHs(mol) AllChem.EmbedMolecule(mol) AllChem.MMFFOptimizeMolecule(mol) # Write out Gaussian input file file_name = smiles + '.com' with open(file_name, 'w') as f: f.write('%chk=' + smiles + '.chk\n') f.write('%nprocshared=4\n') f.write('%mem=4GB\n') f.write('#p opt b3lyp/6-31g(d) empiricaldisper-sion=gd3\n\n') f.write(smiles + '\n\n') f.write('0 1\n') for atom in mol.GetAtoms(): pos = mol.GetConformer().GetAtomPosition(a-tom.GetIdx()) f.write('{:<4} {:>10.5f} {:>10.5f} {:>10.5f}\n'.format(a-tom.GetSymbol(), pos.x, pos.y, pos.z)) f.write('\n\n') return file_name

D

C

# this function downloads a pdb file from the web and saves it to the current directorydef download_pdb(pdb_id): pdb_file = pdb_id + ".pdb" url = "https://files.rcsb.org/download/" + pdb_file response = urllib.request.urlopen(url) data = response.read() pdb_file = open(pdb_file, 'wb') pdb_file.write(data) pdb_file.close() return pdb_file

# download the structure of 1lyspdb_id = "1lys"pdb_file = download_pdb(pdb_id)

# now compute the radius of gyration of 1lys with mdtraj# load the pdb file into mdtrajtraj = md.load(pdb_file.name)# compute the radius of gyrationrg = md.compute_rg(traj)print(rg)

FIG. 3. Example prompts and either resulting code (B,D), or final figures that emerged from running the resulting code (A,C)(full details in the SI). Examples are in Python because our prompts include characteristics of Python code comments, butCodex can work in nearly programming language included in its corpus.

Codex is not always successful. However, the rapid paceof progress in this field shows that we should begin tothink seriously about these tasks being solved.

Will using code from Codex make chemists better orworse programmers? We think better. Codex removesthe tedium of programming and lets chemists focus thehigh-level science enabled with programs. Furthermore,the process of creating a prompt string, mentally check-ing whether it seems reasonable, testing that code on asample input, and then iterating by breaking down theprompt string into simpler tasks will result in better al-gorithmic thinking by chemists. The code generated, ifnot guaranteed to be correct, at least satisfies commonsoftware coding conventions with clear variable names,and typically employs relevant software libraries to sim-plify complex tasks. We ourselves have learned about anumber of existing chemistry software libraries that wewould not have discovered otherwise through our itera-tive prompt creation.

Classroom settings. We and many of our colleaguesaround the world have begun introducing programmingassignments as a component of our courses (especially inphysical chemistry) [23]; this has dual pedagogical pur-poses of reinforcing the physical meaning underlying the

equations we scribble on the board, and teaching our stu-dents a skill that is useful both for research and on thejob market. One of us has even written a book on deeplearning in chemistry and materials science based aroundthis concept [24]. But will code generation models re-sult in poor academic honesty, especially when standardproblems can be solved in a matter of seconds (Fig. 3)?Realistically we have few methods to police our students’behavior in terms of collaborating on programming as-signments or copying from web resources. We rely, atleast in part, on their integrity. We should rethink howthese assignments are structured. Firstly, we currentlylimit the difficulty of programming assignments to alignwith the median programming experience of a studentin our course. Perhaps now we can move towards moredifficult and compound assignments. Secondly, we canmove towards thinking of these assignments as a labora-tory exercise, where important concepts can be exploredusing the software rather than concentrating on the pro-cess of programming itself. Lastly, our coursework andexpectations should match the realities of what our stu-dents will face in their education and careers. They willalways have access to web resources and, now, tools likeCodex. We should embrace the fact that we no longer

4

need to spend hours on teaching loops and instead focuson chemistry.

Ongoing challenges

Access and price. Currently, access to advanced mod-els from OpenAI and tools like GitHub copilot are limitedto users accepted into an early tester program. Pricingfrom the GPT-3 model by OpenAI indicates a per-querycost that is directly proportional to the length of the in-put prompt, typically on the order of 1-3 cents per query.This model may of course change, but it is reasonable toexpect that Codex will not be free until either there arecompeting open-source models or the hardware requiredfor inference drops in price. Depending on this cost struc-ture, these commercial NLP models may be inaccessibleto the academic community, or to all but the most fundedresearch groups and universities. For example, a groupmight need to run hundreds of thousands of queries toparse through academic literature and tens of thousandsfor students in a medium size course, and these wouldcertainly be cost prohibitive. Models developed by theopen source community currently lag commercial ones inperformance, but are freely usable, and will likely be thesolution taken up in many areas of academia. However,even these models require access to significant computa-tional resources to store and execute the models locally,and so we encourage the deployment of these models byresearchers who have such computational resources in away in which they can be equitably available.

Correctness. Code generation models do not guaran-tee correctness. Codex typically generates correct code atabout a 30% rate on a single solution on standard prob-lems, but improves to above 50% if multiple solutionsare tried [1]. In practice, we find that mistakes occurwhen a complex algorithm is requested with little clar-ity. Iterating by breaking a prompt into pieces, chainingtogether prompts into a dialogue, and giving additionalclues like a function signature or imports usually yields asolution. The code generated rarely has syntax mistakes,but we find it fails in obvious ways. Over-reliance on AI-generated code without careful verification could resultin a loss of trust in scientific software and the analysisperformed in published works. However, this is alreadyan issue in scientific programming and strategies to as-sess correctness of code apply equally to human and AI-generated code. Interestingly, Codex can generate unittests for code, although it is not clear that this strategycan identify its own mistakes.

Fairness/bias. As discussed in the Codex paper [1],there are a number of possible issues related to fairnessand bias which could accrue over time. The use of AI gen-erated code, and then the updated training of that AI onthe new code, could lead to a focus on a narrow range ofpackages, methods, or programming languages. For ex-ample, Python is already pushing out other programminglanguages in computational chemistry and this could in-

crease due to the performance of Codex in Python overlanguages like Fortran or Julia. Another example we no-ticed is the preference of Codex to generate code usingcertain popular software libraries, which could lead toconsolidation of use. For example, a single point energycalculation shown in the SI selects the package Psi4 if themodel is not prompted to use a particular software.

Outlook

Bench chemists have expressed the fear that automa-tion will reduce the need for synthetic hands in the lab[25]. Now it looks like these NLP models could reducethe need for computational chemists even sooner. Wedisagree in both cases. Better tools have not reduced theneed for scientists over time, but rather expanded thecomplexity of challenges that can be tackled by a singlescientist or a team in a given amount of time. Despitethe challenges in the previous section, we foresee the useof NLP models in chemistry increasing accessibility ofsoftware tools, and greatly increasing the scope of whata single research group can accomplish.

Acknowledgements

Research reported in this work was supported bythe National Institute of General Medical Sciences ofthe National Institutes of Health under award numberR35GM137966 (to ADW) and R35GM138312 (to GMH).

∗ hockyg@nyu.edu† andrew.white@rochester.edu

[1] M. Chen, J. Tworek, H. Jun, Q. Yuan, H. Ponde, J. Ka-plan, H. Edwards, Y. Burda, N. Joseph, G. Brockman,et al., Evaluating large language models trained on code,arXiv:2107.03374 (2021).

[2] Q. Sun, T. C. Berkelbach, N. S. Blunt, G. H. Booth,S. Guo, Z. Li, J. Liu, J. D. McClain, E. R. Sayfutyarova,S. Sharma, et al., Pyscf: the python-based simulations ofchemistry framework, Wiley Interdiscip. Rev. Comput.Mol. Sci. 8, e1340 (2018).

[3] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit,L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin,Attention is all you need, in Advances in neural informa-tion processing systems (2017) pp. 5998–6008.

[4] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, Bert:Pre-training of deep bidirectional transformers for lan-guage understanding, arXiv:1810.04805 (2018).

[5] More generally, “tokens” are masked.[6] W. L. Taylor, “cloze procedure”: A new tool for measur-

ing readability, Journalism quarterly 30, 415 (1953).[7] A. M. Dai and Q. V. Le, Semi-supervised sequence learn-

ing, Advances in neural information processing systems28, 3079 (2015).

[8] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei,I. Sutskever, et al., Language models are unsupervisedmultitask learners, OpenAI blog 1, 9 (2019).

[9] I. Sutskever, J. Martens, and G. E. Hinton, Generatingtext with recurrent neural networks, in ICML (2011).

5

[10] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Ka-plan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry,A. Askell, et al., Language models are few-shot learners,arXiv preprint arXiv:2005.14165 (2020).

[11] T. Hueckel, G. M. Hocky, J. Palacci, and S. Sacanna,Ionic solids from common colloids, Nature 580, 487(2020).

[12] M. Krallinger, F. Leitner, O. Rabal, M. Vazquez,J. Oyarzabal, and A. Valencia, Chemdner: The drugs andchemical names extraction challenge, Journal of chemin-formatics 7, 1 (2015).

[13] Unpublished, but part of ongoing work known asdavinci-instruct GPT-3 variant.

[14] L. Gao, S. Biderman, S. Black, L. Golding, T. Hoppe,C. Foster, J. Phang, H. He, A. Thite, N. Nabeshima,et al., The pile: An 800gb dataset of diverse textfor language modeling, arXiv preprint arXiv:2101.00027(2020).

[15] E. M. Bender, T. Gebru, A. McMillan-Major, andS. Shmitchell, On the dangers of stochastic parrots: Canlanguage models be too big?, in Proceedings of the 2021ACM Conference on Fairness, Accountability, and Trans-parency (2021) pp. 610–623.

[16] I. Solaiman and C. Dennison, Process for adapting lan-guage models to society (palms) with values-targeteddatasets, arXiv preprint arXiv:2106.10328 (2021).

[17] L. Reynolds and K. McDonell, Prompt programming forlarge language models: Beyond the few-shot paradigm,in Extended Abstracts of the 2021 CHI Conference onHuman Factors in Computing Systems (2021) pp. 1–7.

[18] Https://github.com/whitead/marvis.[19] U. Raucci, A. Valentini, E. Pieri, H. Weir, S. Seritan,

and T. J. Mart́ınez, Voice-controlled quantum chemistry,Nat. Comp. Sci. 1, 42 (2021).

[20] W. Humphrey, A. Dalke, and K. Schulten, Vmd: visualmolecular dynamics, J. Mol. Graph. 14, 33 (1996).

[21] M. K. MacLeod and T. Shiozaki, Communication: Auto-matic code generation enables nuclear gradient computa-tions for fully internally contracted multireference theory,J. Chem. Phys. 142, 051103 (2015).

[22] T. Zirwes, F. Zhang, J. A. Denev, P. Habisreuther, andH. Bockhorn, Automated code generation for maximizingperformance of detailed chemistry calculations in open-foam, in High Performance Computing in Science andEngineering’17 (Springer, 2018) pp. 189–204.

[23] A. Ringer McDonald, Teaching programming across thechemistry curriculum: A revolution or a revival?, inTeaching Programming across the Chemistry Curriculum(ACS Publications, 2021) pp. 1–11.

[24] A. D. White, Deep Learning for Molecules and Materials(2021).

[25] Chemjobber, Will robots kill chemistry?, Chem. Eng.News 97 (2019).

6

Supporting Information

Unless otherwise specified, the davinci-codex model was used with T = 0.

Dissociation energy

Here, T = 0.1 for each case.Result 1 (used to make Fig. 1):

"""

Compute the dissociation curve of H2 using the pyscf library

"""

import numpy as np

import matplotlib.pyplot as plt

from pyscf import gto, scf

# Define the molecule

mol = gto.M(atom=’H 0 0 0; H 0 0 1.4’, basis=’sto-3g’)

# Perform the SCF calculation

scf_eng = scf.RHF(mol)

scf_eng.conv_tol = 1e-9

scf_eng.conv_tol_grad = 1e-7

scf_eng.kernel()

# Plot the dissociation curve

dissociation_curve = np.arange(0.5, 3.0, 0.1)

energies = []

for r in dissociation_curve:

mol = gto.M(atom=’H 0 0 0; H 0 0 ’.format(r), basis=’sto-3g’)

scf_eng = scf.RHF(mol)

scf_eng.conv_tol = 1e-9

scf_eng.conv_tol_grad = 1e-7

scf_eng.kernel()

energies.append(scf_eng.e_tot)

plt.plot(dissociation_curve, energies)

plt.xlabel(’Length of bond (Angstrom)’)

plt.ylabel(’Total energy (Hartree)’)

plt.show()

Result 2

"""

Compute the dissociation curve of H2 using the pyscf library

"""

import numpy as np

import pyscf

from pyscf import gto, scf, dft

# Define the molecule

mol = gto.M(atom=’H 0 0 0; H 0 0 1.4’, basis=’sto-3g’, verbose=3)

# Compute the reference energy

en_ref = mol.energy_nuc()

# Compute the dissociation curve

r = np.linspace(0.5, 2.0, 100)

en = []

for b in r:

mol.atom = [[’H’, (0, 0, 0)], [’H’, (0, 0, b)]]

mol.build()

7

mf = scf.RHF(mol)

mf.kernel()

en.append(mf.e_tot - en_ref)

# Plot the dissociation curve

import matplotlib.pyplot as plt

plt.plot(r, en)

plt.xlabel(’Bond length (Bohr)’)

plt.ylabel(’Energy (Hartree)’)

plt.show()

Result 3:

"""

Compute the dissociation curve of H2 using the pyscf library

"""

import numpy as np

import pyscf

from pyscf import gto, scf, dft, cc, mp

from pyscf.geomopt.berny_solver import optimize

mol = gto.Mole()

mol.atom = """

H 0.0 0.0 0.0

H 0.0 0.0 0.75

"""

mol.basis = "6-31G"

mol.verbose = 0

mol.build()

mf = scf.RHF(mol)

mf.kernel()

# Optimize the geometry of the molecule

mol_eq = optimize(mp.MP2(mf))

# Compute the dissociation curve

r_array = np.arange(0.6, 4.0, 0.1)

e_array = []

for r in r_array:

mol_eq.atom = [[’H’, (0, 0, 0)], [’H’, (0, 0, r)]]

mol_eq.build()

e_array.append(mol_eq.kernel())

8

Caffeine from SMILES

Result 1: Used to make Fig. 3(A)

"""

1. Load the caffeine molecule from SMILES

2. Draw it

"""

from rdkit import Chem

from rdkit.Chem import Draw

from rdkit.Chem.Draw import IPythonConsole

IPythonConsole.ipython_useSVG = True

caffeine = Chem.MolFromSmiles(’CN1C=NC2=C1C(=O)N(C(=O)N2C)C’)

Draw.MolToImage(caffeine)

Result 2:

"""

1. Load the caffeine molecule from SMILES

2. Draw it

"""

from rdkit import Chem

from rdkit.Chem import Draw

from rdkit.Chem.Draw import IPythonConsole

from rdkit.Chem import Descriptors

# Load the caffeine molecule from SMILES

mol = Chem.MolFromSmiles(’CN1C=NC2=C1C(=O)N(C(=O)N2C)C’)

# Draw it

Draw.MolToImage(mol)

9

2D random walk

Full example from Fig. 3(C). The first part produces the image on the left. The second follow up request to write afunction produces the figure on the right, followed by other trials that don’t necessarily do exactly what the generatedcomments say.

"""This script generates the behavior of a 2D random walk and plots trajectories"""import numpy as npimport matplotlib.pyplot as plt

# ParametersN = 1000 # number of stepsx = np.zeros(N) # x positiony = np.zeros(N) # y position

# Random walkfor i in range(1, N):

r = np.random.rand()if r < 0.25:

x[i] = x[i-1] + 1y[i] = y[i-1]

elif r < 0.5:x[i] = x[i-1] - 1y[i] = y[i-1]

elif r < 0.75:x[i] = x[i-1]y[i] = y[i-1] + 1

else:x[i] = x[i-1]y[i] = y[i-1] - 1

# Plotplt.plot(x, y)plt.show()

"""Rewrite this into a function"""def random_walk(N):

x = np.zeros(N)y = np.zeros(N)for i in range(1, N):

r = np.random.rand()if r < 0.25:

x[i] = x[i-1] + 1y[i] = y[i-1]

elif r < 0.5:x[i] = x[i-1] - 1y[i] = y[i-1]

elif r < 0.75:x[i] = x[i-1]y[i] = y[i-1] + 1

else:x[i] = x[i-1]y[i] = y[i-1] - 1

return x, y

# Plotx, y = random_walk(N)plt.plot(x, y)plt.show()

# Plot multiple trajectoriesfor i in range(10):

x, y = random_walk(N)plt.plot(x, y)

plt.show()

# Plot multiple trajectories with different colorsfor i in range(10):

x, y = random_walk(N)plt.plot(x, y, color=’black’)

plt.show()

# Plot multiple trajectories with different colors and markersfor i in range(10):

x, y = random_walk(N)plt.plot(x, y, color=’black’, marker=’o’)

plt.show()

10

Chemical entity recognition example from Fig 2

GPT-3 was used with T=0.7

This code extracts chemical entities:

Sentence: Negatively charged PS particles were produced in the same fashion, replacing

2,2’-azobis(2-methylpropionamidine) dihydrochloride with an equivalent weight amount of

potassium persulfate (≥99% from MilliporeSigma) Chemical Entities:

2,2’-azobis(2-methylpropionamidine) dihydrochloride , potassium persulfate

---

Sentence: Degassed CHCl3 (250 mL) was added via canula and the solution was brought to

reflux for 10 hours.

Chemical Entities: CHCl3

---

Sentence: The solution was washed with 1 M NaOH and dried with MgSO4.

Chemical Entities: NaOH, MgSO4

---

Sentence: In brief, fluorescent dyes dissolved in tetrahydrofuran (THF) were added to

particles stabilized with Pluronic F108 (MilliporeSigma) to a final concentration of 30% v/v

THF, then diluted by a factor of five before washing the particles via multiple sedimentation

and resuspension cycles to set them in pure water.

Chemical Entities: Pluronic F108, tetrahydrofuran, THF

11

Electronic structure input file generation

Multiple trials at generating input files with Gaussian, and one without Gaussian specified.

def gauss(smiles):

"""

This function prepares a Gaussian input file from the given smiles using rdkit to prepare initial coordintes by

1. Converting SMILES to molecule

2. Preparing 3D coordinates

3. Writing out Gaussian input file

"""

# Convert SMILES to molecule

mol = Chem.MolFromSmiles(smiles)

# Generate 3D coordinates

mol = Chem.AddHs(mol)

AllChem.EmbedMolecule(mol)

AllChem.MMFFOptimizeMolecule(mol)

# Write out Gaussian input file

file_name = smiles + ’.com’

with open(file_name, ’w’) as f:

f.write(’%chk=’ + smiles + ’.chk\n’)f.write(’%nprocshared=4\n’)f.write(’%mem=4GB\n’)f.write(’#p opt b3lyp/6-31g(d) empiricaldispersion=gd3\n\n’)f.write(smiles + ’\n\n’)f.write(’0 1\n’)for atom in mol.GetAtoms():

pos = mol.GetConformer().GetAtomPosition(atom.GetIdx())

f.write(’:<4 :>10.5f :>10.5f :>10.5f\n’.format(atom.GetSymbol(), pos.x, pos.y, pos.z))

f.write(’\n\n’)return file_name

def gauss(smiles):

"""

This function prepares a Gaussian input file from the given smiles using rdkit to prepare initial coordintes by

1. Converting SMILES to molecule

2. Preparing 3D coordinates

3. Writing out Gaussian input file

"""

# Convert SMILES to molecule

mol = Chem.MolFromSmiles(smiles)

# Generate 3D coordinates

mol = Chem.AddHs(mol)

AllChem.EmbedMolecule(mol, useExpTorsionAnglePrefs=True, useBasicKnowledge=True)

# Write out Gaussian input file

file_name = smiles + ’.com’

file = open(file_name, ’w’)

file.write(’%mem=5gb\n’)file.write(’%nproc=8\n’)file.write(’#p opt b3lyp/6-31g(d) empiricaldispersion=gd3\n’)file.write(’\n’)file.write(’Title Card Required\n’)file.write(’\n’)file.write(’0 1\n’)for atom in mol.GetAtoms():

file.write(str(atom.GetAtomicNum()) + ’ ’ + str(atom.GetX()) + ’ ’ + \str(atom.GetY()) + ’ ’ + str(atom.GetZ()) + ’\n’)

file.write(’\n’)file.close()

return file_name

def run_gauss(file_name):

"""

This function runs Gaussian on the given input file and returns the energy

"""

# Run Gaussian

os.system(’g09 < ’ + file_name + ’ > ’ + file_name + ’.log’)

# Read energy from Gaussian output file

file = open(file_name + ’.log’, ’r’)

12

lines = file.readlines()

file.close()

for line in lines:

if ’EUMP2 =’ in line:

energy = float(line.split()[2])

return energy

def get_energy(smiles):

"""

This function returns the energy of the given molecule

"""

file_name = gauss(smiles)

energy = run_gauss(file_name)

return energy

def single_point(smiles):

"""

This function prepares a quantum chemistry input file from the given smiles using rdkit to prepare initial coordintes

"""

# Generate the 3D structure of the molecule

mol = Chem.AddHs(Chem.MolFromSmiles(smiles))

AllChem.EmbedMolecule(mol)

AllChem.MMFFOptimizeMolecule(mol)

# Write the coordinates to a file

with open(’coords.xyz’, ’w’) as f:

f.write(str(mol.GetNumAtoms()) + ’\n\n’)for atom in mol.GetAtoms():

pos = mol.GetConformer().GetAtomPosition(atom.GetIdx())

f.write(’ :10.4f :10.4f :10.4f\n’.format(atom.GetSymbol(), pos.x, pos.y, pos.z))

# Run the quantum chemistry calculation

subprocess.call([’psi4’, ’coords.xyz’, ’-o’, ’output.dat’])

# Read the output file

with open(’output.dat’, ’r’) as f:

lines = f.readlines()

# Get the energy

energy = float(lines[-1].split()[1])

# Get the coordinates

coords = []

for line in lines[-(mol.GetNumAtoms() + 2):-2]:

words = line.split()

coords.append([float(words[1]), float(words[2]), float(words[3])])

return energy, coords

def get_energy(smiles):

"""

This function returns the energy of the molecule in kcal/mol

"""

energy, _ = single_point(smiles)

return energy * 627.509