abstract: The ubiquitous adoption of Large Language Generation Models (LLMs) in programming has underscored the importance of differentiating between human-written code and code generated by intelligent models. This paper specifically aims to distinguish code generated by ChatGPT from that authored by humans. Our investigation reveals disparities in programming style, technical level, and readability between these two sources. Consequently, we develop a discriminative feature set for differentiation and evaluate its efficacy through ablation experiments. Additionally, we devise a dataset cleansing technique, which employs temporal and spatial segmentation, to mitigate the dearth of datasets and to secure high-caliber, uncontaminated datasets. To further enrich data resources, we employ "code transformation," "feature transformation," and "feature customization" techniques, generating an extensive dataset comprising 10,000 lines of ChatGPT-generated code. The salient contributions of our research include: proposing a discriminative feature set yielding high accuracy in differentiating ChatGPT-generated code from human-authored code in binary classification tasks; devising methods for generating extensive ChatGPT-generated codes; and introducing a dataset cleansing strategy that extracts immaculate, high-grade code datasets from open-source repositories, thus achieving exceptional accuracy in code authorship attribution tasks.
author: Ke Li; Sheng Hong; Cai Fu; Yunhe Zhang; Ming Liu
title: Deciphering the Code: Distinguishing ChatGPT-Generated Code from Human-authored Code through Discriminative Feature Analysis and Dataset Optimization

# Introduction

Since its introduction in November 2022, OpenAI's ChatGPT has become the cynosure in numerous fields, generating palpable enthusiasm. By capitalizing on human feedback reinforcement learning (RLHF) for fine-tuning and judiciously curated datasets, ChatGPT exhibits exemplary capabilities across a plethora of challenging natural language processing (NLP) tasks. These include code synthesis via natural language, text summarization, and the creation of stylistic narratives based on designated elements, in addition to its adeptness in conventional NLP tasks such as translation and text categorization. Moreover, ChatGPT demonstrates the prudence of refraining from responding to inquiries that exceed its knowledge base, contravene ethical norms, or broach sensitive political topics.

ChatGPT's prowess, particularly in facilitating programming through natural language, has engendered significant intrigue. However, reservations concerning the safety and legality of employing intelligent programming assistants like ChatGPT have been voiced by the scholarly community. The opaque nature of ChatGPT's training data spawns uncertainties regarding the provenance of its generated code snippets and the possibility of their harboring security vulnerabilities or unsafe code fragments, which could entangle developers in copyright disputes and code security quandaries.

For example, Stack Overflow, a renowned platform for programming inquiries, has imposed a temporary restraining on ChatGPT-generated content due to its low accuracy which did not match the high quality demanded by users. Concurrently, academic institutions face dilemmas in assessment settings where students might exploit ChatGPT to accomplish programming assignments, thus creating hurdles in evaluating their true acumen and knowledge.

Addressing these issues hinges on the adept discernment of human-written code versus code generated by intelligent models like ChatGPT. This entails two facets: first, the extraction of discriminative features between human-authored and ChatGPT-synthesized code. Conventional feature sets for code author attribution, which are primarily geared toward distinguishing among human authors, may fall short in capturing subtle distinctions, such as code structuring traits, when discerning between ChatGPT and human-authored code. This necessitates ablation studies to pinpoint a discriminative feature set capable of enhancing classification accuracy.

Through ablation studies, we identified variances in programming style, technical level, and readability between ChatGPT-generated and human-authored code. Building upon existing research in code authorship attribution, we formulate a discriminative feature set that enables distinction between these code sources, thereby surmounting feature analysis challenges.

The second facet concerns the compilation of an associated high-quality dataset. Our examination ascertained that prevailing research on code authorship attribution relies on datasets of heterogeneous quality and sources, bereft of standardized benchmarks. Moreover, datasets specifically comprising ChatGPT-generated code are scant owing to its recent advent.

To redress this dataset deficiency, we conceive a dataset cleansing technique grounded in temporal and spatial segmentation. This technique facilitates the procurement of untainted, high-quality datasets from open-source repositories while confirming data authenticity and eliminating extraneous factors related to authorship. Additionally, we employ a semi-automated approach to generate an expansive dataset, with ChatGPT synthesizing millions of lines of code via three strategies: "code transformation," "function transformation," and "custom functionality." This enriched dataset, which amalgamates human-authored and ChatGPT-generated code, furnishes robust data support for our research.

In summation, our research offers the following seminal contributions:

1.  We devise a discriminative feature set predicated on heuristic code segmentation, which efficaciously differentiates ChatGPT-generated code from human-authored code, achieving a classification accuracy of over 90% in code origin identification tasks through synergistic use with machine learning algorithms.

2.  We introduce three strategies to amass a copious volume of ChatGPT-generated code: "code transformation," "function transformation," and "function customization."

3.  We put forth a dataset cleansing technique centered on temporal and spatial segmentation, facilitating the extraction of immaculate, high-quality code datasets from open-source repositories. Employing this technique, we have amassed and purified approximately 20GB of code datasets. Subsequent application of these datasets to code author attribution tasks yielded an impressive accuracy rate of no less than 95%.

Our contributions establish a robust theoretical framework for distinguishing code generated by advanced models like ChatGPT from human-authored code. These findings bear considerable practical implications, including promoting academic integrity, protecting intellectual property, and bolstering software security, while fostering the sustainable development of AI in programming. This research is poised to catalyze advancements in related domains and pave the way for a harmonious coexistence between human developers and intelligent programming assistants.

The remainder of this paper is organized as follows: Section 2 presents research work related to AI-generated content detection, code authorship attribution and ChatGPT-generated code. Section 3 provides an overview of the creation of datasets comprising human-generated code and ChatGPT-synthesized code. Section 4 elucidates the methodologies employed in developing the discriminative feature set. Section 5 details the experiments conducted and the ensuing analysis of the results. Section 6 concludes the paper and provides an outlook for further research. Finally, Section 7 outlines the limitations of our research and suggests potential directions for future research.

# Related Work

## AIGC Detector

Research is already emerging in the field of detecting and identifying content generated by artificial intelligence (AI) models. For instance, in the domain of natural language processing, there have been notable studies focused on distinguishing AI-generated content from human-created content.

Mitrovi´c et al. employed a machine learning approach to differentiate ChatGPT by extracting features from text messages. Their study achieved an accuracy of 79% by focusing on shorter text responses generated by ChatGPT in comparison to manually generated text.

In another study, Guo et al. collected 40K questions and corresponding answers from both human experts and ChatGPT to construct the Human ChatGPT Comparison Corpus (HC3) dataset. This dataset encompassed various domains, including open domain, finance, medical, legal, and psychological fields. Through the HC3 dataset, the researchers investigated differences between the features of ChatGPT-generated texts and those of human experts. They conducted feature analysis from multiple perspectives, including Vocabulary Features, Part-of-Speech and Dependency Analysis, and Sentiment Analysis. Additionally, they employed classical machine learning and deep learning techniques, performing experiments with three representative methods to further validate their findings.

Wenxiong Liao et al. conducted a comprehensive analysis of medical texts to distinguish between content written by human experts and text generated by ChatGPT. Their study involved constructing a specialized dataset for medical texts and analyzing linguistic features, such as vocabulary, among others. To detect medical texts generated by ChatGPT, they employed a machine learning approach, specifically utilizing a BERT-based model. The results were promising, with the model achieving an impressive F1 score exceeding 95%. Furthermore, the researchers observed that medical texts authored by humans tend to be more concrete, diverse, and contain a wealth of useful information. Conversely, medical texts generated by ChatGPT prioritize fluency and logic, often expressing general terminologies rather than providing context-specific, problem-related information.

## Code authorship attribution

To date, there has been limited academic research addressing the specific problem of distinguishing code generated by AI models from code written by humans. However, this research can be aligned with the traditional code authorship attribution problem, which focuses on extracting code-level features to differentiate or trace the authorship of code.

In the field of code authorship attribution, classical machine learning approaches have been explored. Caliskan-Islam et al. utilized a dataset comprising the code of 1600 authors from Google Code Jam. They extracted approximately 120,000 features based on vocabulary, layout, and syntax from the source code. By employing a random forest model consisting of 300 decision trees, they achieved up to 98% accuracy on a test set comprising code from 250 programmers who participated in Google Code Jam 2014, effectively pushing the boundaries of machine learning models.

Regarding deep learning, Abuhamad et al. employed a dataset consisting of 1600 authors from Google Code Jam (spanning from 2008 to 2016) and 1987 repositories from GitHub, with 142 C++ and 745 C programmers. They initially utilized the text analysis tool TF-IDF (Term Frequency-Inverse Document Frequency) for preprocessing the source code, which served as input for a deep learning network. They designed a deep learning framework based on recurrent neural networks to extract code features. The author attribution task was then accomplished using a random forest classifier. This approach achieved an accuracy of 96% in the Google Code Jam experiment with 1600 authors and 94.38% on the real dataset of 745 C programmers.

## Code generated by ChatGPT

The current research on code generated by ChatGPT encompasses several areas, including code security, code correctness, code quality improvement, code error resolution, code meaning interpretation, and the potential of ChatGPT as a programming assistant.

Raphaël Khoury et al. explored the capabilities of ChatGPT in generating programs and assessed the security of the resulting source code. They also investigated the effectiveness of prompting ChatGPT to enhance code security and delved into the ethical considerations associated with leveraging AI for code generation. The findings indicate that ChatGPT demonstrates some awareness of potential vulnerabilities. However, it frequently generates source code that lacks robustness against certain attacks.

Jiawei Liu et al. introduced EvalPlus, a code synthesis benchmarking framework designed to thoroughly evaluate the functional correctness of code synthesized by Language Models (LLMs). In their work, they extended the widely used HUMANEVAL benchmark and created HUMANEVAL+, which includes an additional 81× generated tests. Through extensive evaluation across 14 popular LLMs, including GPT-4 and ChatGPT, they demonstrated that HUMANEVAL+ effectively detects a significant number of previously undetected erroneous code synthesized by LLMs. On average, it reduces the pass@k metric by 15.1%.

The research conducted by Burak Yetiştiren et al. aims to conduct a comparative analysis of prominent code generation tools, including GitHub Copilot, Amazon CodeWhisperer, and ChatGPT, in terms of various code quality metrics such as Code Validity, Code Correctness, Code Security, Code Reliability, and Code Maintainability. To achieve this, they utilize the benchmark HumanEval Dataset to evaluate the generated code based on the proposed code quality metrics.The analysis reveals that the latest versions of ChatGPT, GitHub Copilot, and Amazon CodeWhisperer achieve correct code generation rates of 65.2%, 46.3%, and 31.1% respectively. Moreover, the newer versions of GitHub Copilot and Amazon CodeWhisperer demonstrate improvement rates of 18% and 7% respectively in terms of generating correct code. Additionally, the average technical debt, considering code smells, is found to be 8.9 minutes for ChatGPT, 9.1 minutes for GitHub Copilot, and 5.6 minutes for Amazon CodeWhisperer.

Jules White et al. explores several prompt patterns that have been applied to improve requirements elicitation, rapid prototyping, code quality, refactoring, and system design.

Dominik Sobania et al. evaluate the bug fixing performance of ChatGPT on the widely used QuixBugs benchmark set. They compare its performance with several other approaches reported in the literature. The findings reveal that ChatGPT's bug fixing capability is on par with common deep learning approaches like CoCoNut and Codex, and significantly outperforms standard program repair methods. Moreover, by providing hints to ChatGPT, they were able to further enhance its success rate, with ChatGPT successfully fixing 31 out of 40 bugs, surpassing the state-of-the-art performance.

Eason Chen et al. presents GPTutor, a ChatGPT-powered rogramming tool, which is a Visual Studio Code extension using the ChatGPT API to provide programming code explanations. By integrating Visual Studio Code API, GPTutor can comprehensively analyze the provided code by referencing the relevant source codes. Preliminary evaluation indicates that GPTutor delivers the most concise and accurate explanations compared to vanilla ChatGPT and GitHub Copilot. Moreover, the feedback from students and teachers indicated that GPTutor is user-friendly and can explain given codes satisfactorily.

Haoye Tian et al. conduct an empirical analysis to evaluate the capabilities of ChatGPT as a fully automated programming assistant, with a focus on code generation, program repair, and code summarization. The study specifically examines ChatGPT's performance in solving common programming problems and compares it to state-of-the-art approaches using two benchmark datasets. The findings demonstrate that ChatGPT effectively addresses typical programming challenges. However, the researchers also identify limitations in its attention span. They observe that when faced with comprehensive problem descriptions, ChatGPT's focus can be constrained, thus hindering its ability to leverage its vast knowledge for effective problem-solving.

Yihong Dong et al. present a self-collaboration framework for code generation employing LLMs, exemplified by ChatGPT. Specifically, through role instructions, 1) Multiple LLMs act as distinct "experts", each responsible for a specific subtask within a complex task; 2) Specify the way to collaborate and interact, so that different roles form a virtual team to facilitate each other's work, ultimately the virtual team addresses code generation tasks collaboratively without the need for human intervention. They conduct comprehensive experiments on various code-generation benchmarks. Experimental results indicate that self-collaboration code generation relatively improves 29.9%-47.1% Pass@1 compared to direct code generation, achieving state-of-the-art performance and even surpassing GPT-4.

# Human and ChatGPT Code Dataset

The dataset as a whole can be divided into two parts: those generated by ChatGPT and those crawled from Github.

## ChatGPT Code Dataset

Our endeavor to comprehensively discern the attributes of code generated by ChatGPT necessitated the creation of corresponding datasets. We employed two versions of ChatGPT: GPT-3.5, which avails batch generation through an API, and GPT-4, which presently only facilitates generation through human interaction due to the absence of an open API.

It is imperative to recognize that ChatGPT's training data could encompass code from open-source repositories on Github. To investigate the viability of employing ChatGPT for code creation, and to discern its idiosyncratic features, we applied three distinct strategies: "code translation," "function translation," and "custom functions" for each version of ChatGPT. These strategies are explicated as follows:

-   **Code Translation:** This strategy entails supplying ChatGPT with a code snippet written in programming language X, and tasking it to generate equivalent functionality code in a different programming language Y. The generated code must span at least 100 lines to circumvent the production of "boilerplate code" replete with annotations, which could impair data quality. As the generated code is translated into a different programming language, it is less likely to have been part of ChatGPT's training data, hence, it is relatively novel. But this also runs the risk of mimicking the wording and logic of the original code.

-   **Functional Translation:** In contrast to Code Translation, Functional Translation requires ChatGPT to first analyze and abstract a functional description from a given code snippet. Subsequently, ChatGPT is instructed to generate code in the same programming language as the original snippet based on the derived functional description. This method prevents direct imitation of the original code, but it is slightly less innovative compared to Code Translation since the output is in the same language as the original code, which could be part of ChatGPT's training data.

-   **Functional customisation:** This strategy diverges from the prior two by prompting ChatGPT to generate code based on pre-existing functional descriptions sourced from programming competitions, textbooks, or other typical programming tasks. As this approach neither supplies ChatGPT with sample code nor restricts it to the original programming language, it engenders greater originality compared to Function Translation. Nonetheless, since solutions to these programming tasks might be accessible online and included in ChatGPT's training data, this approach is not as novel as Code Translation.

## Human Code Dataset

Human-authored code datasets are integral to research in the domain of code authorship attribution. Present studies predominantly utilize datasets procured from four principal sources:

1.  *Educational Programming Tasks:* These datasets encompass code penned by students for class assignments. However, their restrictive scope and educational focus impede their generalizability.

2.  *Competitive Programming Archives (e.g., Google Code Jam):* Although these datasets are of high caliber, they might not accurately depict programming methodologies employed in real-world software development, owing to the specialized nature of competition problems and environments.

3.  *Open Source Repositories (e.g., Github):* Being the most reflective of real-world programming practices, these datasets are invaluable. However, ascertaining the sole authorship of the code is challenging due to collaborative projects and the opacity of development processes.

4.  *Textbook Supplements:* Code written by authors to supplement programming textbooks is also used. However, this data is limited by the scope of the textbooks and lacks generalization.

Among these, datasets derived from open-source repositories are ostensibly the most authentic and efficacious for capturing coding styles. Github, being the world's largest open-source community, hosts numerous repositories. Although individuals frequently upload their code, the prevalence of shared or borrowed code is significant. Additionally, coding styles evolve over time, with discernible differences between code written at the nascent stages of learning and that written post-acquiring professional expertise. Consequently, indiscriminate code harvesting from repositories could compromise dataset quality and obfuscate the analysis of distinct coding styles.

To alleviate this issue, this paper introduces a methodology premised on temporal and spatial segmentation to collate code from individual repositories, thereby capturing the essence of an author's coding style during a specific timeframe. The ensuing section delineates the steps undertaken to clean the dataset.

1.  *Initial Cleanup:* In this step, organization accounts, forked repositories, and duplicates are eliminated. This culling is predicated on existing literature and conventions. The filtering of accounts is restricted to individual users, thereby excluding organization accounts. Forked repositories are omitted, as they typically do not contain original code. Moreover, duplicate repositories and files are removed to economize on storage and computation.

2.  *Temporal Segmentation:* Repositories are categorically segmented by their creation year. Studies suggest that an author's coding style remains relatively stable for approximately two years. Consequently, the dataset is divided based on the creation year, ensuring each segment contains code within a specific timeframe. In our experiments, repositories are classified from 2008 to 2022, rendering 15 categories. Subsequently, we focus on the code produced in the recent year (2021-2022).

3.  *Spatial Segmentation:* This step entails the removal of third-party libraries and collaborative repositories. Repositories containing third-party libraries are excluded based on naming conventions. Furthermore, repositories with multiple contributors are identified and removed by analyzing the "contributors" metadata. A comprehensive list of common third-party libraries for C++ and Java was compiled based on frequency analysis and practical development experience, serving as a filter criterion.

In theory, post these cleaning steps, the residual source code should predominantly be authored by the individuals themselves. Nevertheless, some exceptions may include code obtained from external sources. Such instances are considered data noise and are disregarded.

The aforementioned steps ensure that the final dataset chiefly comprises code that reflects the individual authors' distinct coding styles.

# Discriminative Features Set

In this study, we draw upon methodologies from traditional code authorship attribution; however, we tailor the feature extraction process to suit the distinct nature of our task. Traditional code authorship aims to discriminate between code written by different individuals, while our objective is to classify code written by humans as one category and code generated by ChatGPT as another. Consequently, this necessitates a modified approach to feature selection. We adapt the feature selection from traditional code authorship by conducting a heuristic code feature analysis. This enables us to construct a discriminative feature set that effectively discerns between human-authored code and ChatGPT-generated code. This feature set comprises three main categories: lexical features, structural layout features, and semantic features. Notably, this refined feature set diverges from those traditionally employed in code authorship studies and is specifically tailored for our task. In the ensuing subsections, we expound upon the design methods for each category within the feature set.

## Lexical features

Instead of analyzing the lexical features of the entire code, we segregate the vocabulary within the code into four distinct categories:

1.  *Comments and Strings:* This category includes single and multi-line comments, as well as strings enclosed in double quotes. These text blocks are indicative of the author's textual style.

2.  *Identifiers:* Comprising class names, method names, variable names, and interface names, identifiers reveal the author's naming conventions and library usage patterns.

3.  *Keywords:* These reserved words are intrinsic to the programming language, governing syntax structures, control flow, data types, and variable declarations. Analyzing the usage of keywords sheds light on the author's programming practices within the language.

4.  *Imported Libraries:* This category encompasses the standard and third-party libraries incorporated in the code through "include" (C++) or "import" (Java) statements. This reflects the author's familiarity with various libraries.

Prior to lexical analysis, we tokenize the code, taking into account conventions such as camel case or underscores in identifiers. We separate words in comments, strings, and identifiers using spaces and punctuation marks. Subsequently, we split these tokens according to naming conventions and normalize them to lowercase. For keywords, we compare the tokens against a set of language-specific keywords. For imported libraries, we retain the complete names as they represent entities and are indicative of the author's style.

We tally the count of each vocabulary type and compute the term frequency (TF) of each word within these categories.

## Structural Layout Features

In our preliminary analysis of ChatGPT code datasets, we observed that ChatGPT adheres to certain conventional formatting standards regarding layout features. While this is also typical of human-authored code, directly employing layout features from traditional code authorship as distinguishing factors would not be efficacious.

However, upon rigorous comparative analysis, we discerned subtle yet distinguishing layout and structural features that are characteristic of ChatGPT-generated code. We identified 22 such features, encompassing aspects such as comment ratio, blank line ratio, presence of line breaks preceding braces, average nesting depth, indentation length, and the average number of parameters in functions. These features are reflective of coding conventions and styles, and exhibit discernable disparities between human-authored code and ChatGPT-generated code. For a comprehensive listing of these features, refer to Table .

## Semantic features

In this study, we undertook an exhaustive, hands-on comparative analysis supplemented with extensive literature review to devise a set of semantic features intrinsic to code, an aspect that had remained untouched in prior research concerning code authorship attribution. It is our conviction that humans and AI embody distinct proficiencies and constraints when it comes to programming. For instance, human programmers excel in logical reasoning and possess the invaluable ability to collaborate and brainstorm, yet they are shackled by the boundaries of their knowledge and are susceptible to emotional biases such as procrastination or frustration. Contrarily, AI boasts an encyclopedic reservoir of knowledge, and remains impervious to emotional fluctuations; however, it falls short in logical reasoning prowess and lacks the initiative to scrutinize code autonomously. Interestingly, these disparities are mirrored in the semantics of the code written by humans and AI, and these semantics play a pivotal role in various facets like the code's execution efficiency, accuracy in real-world deployment, performance metrics, and so on. After meticulously evaluating various semantic features, we have zeroed in on the following three core aspects:

1.  *Runnability Analysis:* This entails compiling and executing the code to identify compilation or runtime errors.

2.  *Correctness Analysis:* If the code is runnable, we input test cases for algorithmic problems to verify whether the code produces accurate outputs.

3.  *Time-Space Performance Analysis:* For code that correctly solves the problem, we analyze the execution time and memory usage under large-scale test cases.

# Differences between Code Authored by Humans and ChatGPT

In Chapter 3, we presented the three categories of features that we identified as discriminative for distinguishing code produced by ChatGPT from human-authored code. In this chapter, we discuss experiments designed around these feature sets: binary classification, explanatory word frequency analysis, and exploratory analysis based on semantic features.

## Experiment Design

### Binary Classification Experiment

This experiment seeks to ascertain the feasibility of utilizing lexical and layout structural features to distinguish ChatGPT-generated code from human-written code, focusing on C++ and Java. We specifically employ lexical and layout structural features as they are readily quantifiable for machine learning models. For performance evaluation, we employ Accuracy, Precision, Recall, and F1 Score metrics, and also conduct ablation studies to investigate the contributions of each feature set.

### Explanatory Word Frequency Analysis Experiment

This experiment visually represents and statistically analyzes the discrepancies in word usage within C++ and Java code authored by ChatGPT and humans. Specifically, we study the frequency of comments, strings, identifiers, keywords, and imported packages/headers. We contrast the frequencies and consider ChatGPT's documentation and relevant studies for additional context, offering analysis for particular variations.

### Exploratory Semantic Analysis

This experiment investigates the semantic distinctions between code produced by ChatGPT and humans when solving identical programming problems. Due to the intricate nature of semantic feature extraction, and constraints in resources and time, this experiment mainly serves to provide insights and stimulate future research. Specifically, we present ChatGPT with 100 algorithm problems from LeetCode and evaluate various aspects, such as difficulty level, pass rate, executability, correctness, and time-space performance.

## Result Summarization

### Effectiveness of Code Detection

```latex
\begin{table*}[ht]\centering
    \small % 使用较小的字体
    \setlength{\tabcolsep}{4pt} % 调整列间距
    \caption{Comparison of Java code ablation experiment.}
    \label{tab:java-code-ablation}
    \begin{tabularx}{\textwidth}{l *{12}{c}cccccccccccc} % 使用tabularx环境,并设置表格宽度为文本宽度
        \toprule
        & \multicolumn{4}{c}{lexical} & \multicolumn{4}{c}{layout} & \multicolumn{4}{c}{all} \\
        \cmidrule(lr){2-5} \cmidrule(lr){6-9} \cmidrule(lr){10-13}
        Algorithm & Accuracy & Precision & Recall & F-Measure & Accuracy & Precision & Recall & F-Measure & Accuracy & Precision & Recall & F-Measure \\
        \midrule
        Random Forest & 0.949 & 0.952 & 0.949 & 0.958 & 0.950 & 0.951 & 0.950 & 0.950 & 0.960 & 0.961 & 0.960 & 0.960 \\
        SMO & 0.940 & 0.940 & 0.940 & 0.940 & 0.878 & 0.883 & 0.878 & 0.877 & 0.958 & 0.959 & 0.958 & 0.958 \\
        Simple Logistic & 0.951 & 0.952 & 0.951 & 0.951 & 0.958 & 0.959 & 0.958 & 0.958 & 0.969 & 0.969 & 0.969 & 0.969 \\
        J48 & 0.974 & 0.974 & 0.974 & 0.974 & 0.949 & 0.950 & 0.949 & 0.949 & 0.978 & 0.978 & 0.978 & 0.978 \\
        \bottomrule
    \end{tabularx}
\end{table*}
```
```latex
\begin{table*}[ht]\centering
    \small % 使用较小的字体
    \setlength{\tabcolsep}{4pt} % 调整列间距
    \caption{Comparison of C++ code ablation experiments.}
    \label{tab:cpp-code-ablation}
    \begin{tabularx}{\textwidth}{l *{12}{c}cccccccccccc} % 使用tabularx环境,并设置表格宽度为文本宽度
        \toprule
        & \multicolumn{4}{c}{lexical} & \multicolumn{4}{c}{layout} & \multicolumn{4}{c}{all} \\
        \cmidrule(lr){2-5} \cmidrule(lr){6-9} \cmidrule(lr){10-13}
        Algorithm & Accuracy & Precision & Recall & F-Measure & Accuracy & Precision & Recall & F-Measure & Accuracy & Precision & Recall & F-Measure \\
        \midrule
        Random Forest & 0.917 & 0.919 & 0.917 & 0.917 & 0.867 & 0.869 & 0.867 & 0.867 & 0.930 & 0.930 & 0.930 & 0.930 \\
        SMO & 0.900 & 0.902 & 0.900 & 0.900 & 0.863 & 0.864 & 0.863 & 0.863 & 0.924 & 0.927 & 0.924 & 0.924 \\
        Simple Logistic & 0.877 & 0.877 & 0.877 & 0.877 & 0.861 & 0.862 & 0.861 & 0.861 & 0.906 & 0.907 & 0.906 & 0.906 \\
        J48 & 0.884 & 0.886 & 0.884 & 0.884 & 0.796 & 0.796 & 0.796 & 0.796 & 0.890 & 0.892 & 0.890 & 0.890 \\
        \bottomrule
    \end{tabularx}
\end{table*}
```

In the binary classification experiment, we analyzed the individual and combined effects of lexical and layout structural features in differentiating between human and ChatGPT-generated code. The results, as depicted in Table and Table , demonstrate that the combination of both feature sets yields the highest accuracy in classification. This underlines the complementary nature of these features in code classification tasks.

### Characteristic Differences

In this subsection, we delve into the distinctions between ChatGPT-generated code and human-authored code by analyzing comments and strings, identifiers, keywords, and library usages. Figures to depict these aspects, focusing on tokens with discrepancies exceeding 50% and ranking them in descending order based on the difference. Our investigation aims to illuminate the inherent characteristics distinguishing ChatGPT-generated code from human-written code. The following observations can be drawn from the disparity in frequencies of these elements:

1.  ChatGPT generally employs more elaborate and semantically rich terminology, whereas humans often favor concise and simpler words.

2.  Human code frequently exhibits remnants of development iterations, such as commented-out code snippets, which are absent in ChatGPT-generated code. Additionally, ChatGPT does not generate extraneous code, while human code sometimes includes unreferenced variables or methods.

3.  The code generated by ChatGPT adheres more closely to coding standards compared to human-authored code. For instance, in C++, ChatGPT prefers the standard "endl" line terminator over the "\n" newline character.

4.  ChatGPT-generated code typically does not exhibit inter-file dependencies, unlike human-written code, which often has a high degree of coupling between multiple files.

5.  ChatGPT emphasizes readability, employing meaningful words for identifiers, whereas humans may use abbreviations or letters with ambiguous meanings.

6.  ChatGPT is more likely to utilize newer language features. For example, in Java, it favors the "foreach" loop iteration over the traditional "for" loop, and in C++, it makes frequent use of the "auto" keyword for type inference.

7.  ChatGPT-generated code resembles sample or template code sometimes, with comments often suggesting areas requiring custom implementation. In contrast, human code is typically intended for release, and employs logging for debugging instead of console output statements which are common in ChatGPT's code.

### Examining the Originality and Semantic Similarities of ChatGPT-Generated Code

In our exploratory experiments, we sought to investigate if ChatGPT tends to replicate code authored by humans and to analyze the semantic differences between the code produced by humans and ChatGPT. To this end, we assembled a dataset composed of 100 programming tasks along with their respective human-authored solutions and codes. We then employed ChatGPT to generate code solutions for these tasks and performed a comparative analysis of the outcomes.

In the case of programming tasks predating ChatGPT's knowledge base cutoff date, the code generated by ChatGPT exhibited an exceptional executability rate of 100% and a correctness rate of 100%. In contrast, human solutions achieved an average correctness rate of 50% under comparable circumstances. Additionally, ChatGPT's solutions demonstrated superior temporal efficiency in approximately 72.75% of the cases and spatial efficiency in roughly 49.16% of cases compared to human-authored code.

For programming tasks postdating ChatGPT's knowledge base cutoff date, ChatGPT achieved an executability rate of 90.9%. The initial correctness rate of ChatGPT's solutions stood at 41%, falling short of the human average of around 50%. However, through subsequent refinements and iterations informed by follow-up questions, ChatGPT's solutions saw their correctness rate ascend to 63.63%. Regarding temporal efficiency, ChatGPT's solutions outperformed human-authored code in approximately 58.91% of cases. Similarly, in spatial efficiency, ChatGPT's solutions outstripped human-authored code in approximately 42% of cases.

Furthermore, we conducted an in-depth comparison of ChatGPT-generated code against human-written code for the same programming tasks. Among the 100 code solutions, only two instances were observed in which ChatGPT exactly replicated human-authored code. In the majority of cases, ChatGPT manifested semantic similarity to human-authored code, echoing the underlying logic and ideas but employing distinct syntactic constructs and variations.

It is crucial to delve into the implications of ChatGPT's semantic similarity with human-authored code. The manifestation of semantic similarity suggests that ChatGPT is capable of discerning and emulating the core logic underlying human coding practices. While this could streamline the code generation process and uphold code quality, it also raises questions regarding innovation and originality in coding. ChatGPT's inclination to reflect existing coding conventions and logic could, in some instances, stifle novel approaches and solutions. Additionally, ethical considerations such as intellectual property rights and plagiarism need to be taken into account, especially in contexts where code is generated for commercial or proprietary applications. Ensuring that ChatGPT-generated code is sufficiently distinct from existing code, and that it acknowledges and credits foundational sources where necessary, is integral to the responsible and ethical deployment of this technology.

# Conclusion

This paper elucidates the critical distinctions between ChatGPT-generated code and human-authored code, along with their ramifications. The salient conclusions drawn from this investigation are enumerated below:

1.  The discriminative feature set proposed in this study demonstrates exceptional efficacy in discerning ChatGPT-generated code from human-authored code, with an accuracy rate surpassing 90%. This underscores the viability of employing the identified feature set for distinguishing between ChatGPT-generated and human-authored code.

2.  Examination of token frequencies unveils pronounced disparities between ChatGPT-generated and human-authored code in the utilization of specific tokens. These discrepancies are indicative of disparate coding practices and preferences. Moreover, ChatGPT-generated code exhibits semantic variations compared to human-authored code when addressing identical programming tasks. These semantic differences represent varying levels of programming expertise and knowledge reservoirs.

3.  ChatGPT has demonstrated a propensity to reproduce existing code in scenarios where tasks are congruent with its pre-existing knowledge base. However, it is worth noting that this is not a simple replication; ChatGPT often recontextualizes and restructures the code, showcasing its ability to assimilate and adapt human programming expertise.

4.  While ChatGPT is proficient in generating functionally correct and efficient code, the semantic similarity with human-authored code raises questions regarding innovation and the ethical aspects of code generation. Ensuring originality and proper acknowledgment in ChatGPT-generated code is vital, especially in contexts involving proprietary or commercial applications.

The revelations from this study deepen our comprehension of the code generation capabilities of ChatGPT and bring into focus the parallels and distinctions between ChatGPT-generated and human-authored code. These insights are invaluable for diverse applications including code identification, analysis, and the integration of AI-assisted code generation into development workflows. Future inquiries can leverage these insights to bolster the precision and efficiency of code identification mechanisms and to investigate strategies for optimally combining the proficiencies of ChatGPT with human ingenuity. Moreover, additional studies could explore the ethical considerations and frameworks needed to guide the responsible use of AI in code generation.

# Limitations

Notwithstanding the invaluable revelations procured from this investigation, it is imperative to recognize certain constraints that warrant consideration:

1.  The ChatGPT code dataset used in this study is relatively small in size. Due to constraints in time and resources, the collected data is still insufficient and unbalanced across different sources, programming languages, styles, and tasks. To enhance the accuracy and reliability of ChatGPT code analysis and differentiation, a larger and more diverse dataset encompassing a wider range of coding styles and sources is needed.

2.  The characterization of the ChatGPT code dataset is not exhaustive. It should be noted that all ChatGPT-generated code samples collected for this study were generated without any specific prompts or instructions. Therefore, the analysis and conclusions presented in this paper are based on the general programming style and state of ChatGPT. It is important to recognize that using specific prompts or instructions during code generation, such as excluding comments or employing a particular programming approach, may potentially interfere with our feature detection process and challenge the validity of the conclusions drawn in this study.

3.  The ChatGPT code dataset might not fully represent the programming style of large-scale code generation models. Currently, ChatGPT is a generic large-scale language generation model that addresses various domains, including code generation. However, it is not exclusively designed as a dedicated large-scale code generation model. Therefore, the programming characteristics observed in the HCCD dataset may not entirely reflect the programming style and behavior of specialized large-scale code generation models.

These constraints delineate avenues for future inquiries and underscore the necessity for broader and more heterogeneous datasets, exhaustive examination of code generation directives, and exploration of the programming attributes of specialized large-scale code generation models. Redressing these constraints will catalyze a more encompassing understanding of code generation models and foster the evolution of more precise and dependable methodologies for differentiating between code engendered by artificial intelligence and that which is authored by humans. Additionally, ethical considerations surrounding code generation and intellectual property should also be integral components of future studies.

# Appendix

This appendix contains a table that summarizes the features used in the analysis. Each feature is accompanied by a brief description to provide clarity on its role within the study. The table can be found on the following page.

```latex
\begin{table*}
  \caption{Summary of Features Used in Analysis}
  \begin{tabular}{p{5cm} p{10cm}}
      \toprule
      \textbf{Feature Name} & \textbf{Description} \\
      \midrule
      Control Structure Density & Logarithm of the ratio of the count of seven control structure-related keywords (do, else if, if, else, switch, for, while) to file length \\
      Ternary Operator Density & Logarithm of the ratio of the count of ternary operators to file length \\
      Token Density & Logarithm of the ratio of the count of tokens to file length \\
      Comment Density & Logarithm of the ratio of the count of comments to file length \\
      Literal Density & Logarithm of the ratio of the count of literals to file length \\
      Keyword Density & Logarithm of the ratio of the count of keywords to file length \\
      Function Density & Logarithm of the ratio of the count of functions to file length \\
      Maximum Nesting Depth & Maximum nesting depth of control and loop structures \\
      Average Branching Factor & Average number of subtrees per code block \\
      Average Parameters per Function & Average number of parameters in functions \\
      Standard Deviation of Parameter Count & Standard deviation of parameter count in functions \\
      Average Line Length & Average length of lines in the code file \\
      Line Length Standard Deviation & Standard deviation of line lengths in the code file \\
      Macro Density & Logarithm of the ratio of the count of preprocessor macros to file length \\
      Tab Character Density & Logarithm of the ratio of the count of tab characters to file length \\
      Space Character Density & Logarithm of the ratio of the count of space characters to file length \\
      Empty Line Density & Logarithm of the ratio of the count of empty lines to file length \\
      Whitespace Ratio & Ratio of whitespace characters (spaces, tabs, new lines) to non-whitespace characters \\
      New Line Preceding Open Brace & Presence of a new line character before opening braces in code blocks \\
      Leading Indentation Type & Indentation at the beginning of each line using tabs or spaces \\
      Maximum AST Node Depth & Maximum depth of nodes in the Abstract Syntax Tree (AST) \\
      AST Node Bigram Frequencies & Relative frequencies of AST node bigrams \\
      Average AST Node Type Depth & Average depth of nodes of each type in the AST \\
      Keyword Frequencies & Relative frequencies of keywords in the code \\
      Average Code Depth in AST Leaves & Average depth of code in AST leaf nodes \\
      Line Length Frequencies & Frequencies of different line lengths in the code file \\
      \bottomrule
  \end{tabular}
  \label{tab:features}
\end{table*}
```


abstract: We have mapped the total sky brightness at 1465 in two adjacent $60\deg$ declination bands with the portable 5.5-m parabolic reflector of the Galactic Emission Mapping (GEM) project, an on-going international collaboration to survey the radio continuum of the sky in decimeter and centimeter wavelengths. The observations were conducted from two locations, one in the USA and the other in Brazil, using a novel instrumental approach to overcome the well-known shortcomings of survey experiments. Our strategy consists of a 1-rpm rotating dish to circularly scan the sky at $30\deg$ from zenith. The dish uses a rim-halo to re-direct the spillover sidelobes of its backfire helical feed toward the sky and the entire assembly has been enclosed inside a wire mesh ground shield in order to minimize and level out the contamination from the ground. The diffraction characteristics of this set-up have been succesfully modelled and undesired systematic striping across the observed bands has been carefully removed by a baseline propagation method which exploits the time-forward and time-backward intersections of the circular scans. The map displays nearly 300 hours of our best quality data taken with a HPBW of $5.4\deg$ at a sensitivity of 20 mK.
author: C. Tello, T. Villela; G.F. Smoot; S. Torres; M. Bersanelli
title: A New Radio Continuum Survey of the Sky at 1465 MHz between declinations $-52\deg$ and $+68\deg$

=

# Introduction

Today, as we witness the dawn of a precision era in observational cosmology, we are becoming increasingly aware that our present knowledge and understanding of the Galactic foregrounds demand a more thorough underpinning. During the course of this meeting we have learned about the results from the latest experiments for measuring the anisotropy in the Cosmic Microwave Background Radiation (CMBR). Yet, despite their success in ruling out a significant contamination of Galactic origin, there are still opened questions regarding the spatial and spectral distribution of the Galactic constituents. New insights have already improved on this picture as the work of Lazarian on Galactic dust properties and of Reynolds on Galactic free-free mapping through H$\alpha$ association have been able to tell us. Such an effort also lies at the core of the Galactic Emission Mapping (GEM) project in order to reveal a new look of the Galactic synchrotron component, capable of overcoming the well-known shortcomings of the existing surveys; let alone the important implications for the search of spectral distortions in the CMBR.

# The GEM Project at 1465

The GEM project (De Amici et al. 1994; Smoot 1999) was conceived 9 years ago at the XXIst IAU General Assembly in Buenos Aires, Argentina, when an international team of researchers set forth to develop a portable radiotelescope for achieving full-sky coverage between 408 and 10. Since its first deployment in the Owens Valley desert near Bishop, CA, (USA) in 1994 the GEM experiment has operated from 3 other locations (Villa de Leyva in Colombia, Tenerife in the Canary Islands and Cachoeira Paulista in Brazil) and gathered observations at 4 of the 5 planned frequencies: 408 (Torres et al. 1996), 1465 (Tello 1997) and $2.3$`<!-- -->`{=html} (Torres, internal report, 1997). For the purpose of these proceedings I will restrict my attention to the data taken at the Owens Valley and Cachoeira Paulista sites at 1465 which provide a nearly 75% sky-coverage.

GEM's mapping strategy uses a 1-rpm rotating dish to scan the sky around the zenith in $60\deg$ wide circles, which due to the earth's rotation provide complete sky coverage of an equally wide declination band (see Fig. 1). The GEM radiotelescope consists of a 5.5-m dish and is double-shielded against stray radiation from the ground. We use a rim-halo to intercept the spillover sidelobes and a wire mesh fence to further cut down and level out the contamination from the ground. By applying a geometric diffraction model to this experimental setup (Tello et al. 1999), together with measurements of the backfire helical feed pattern and selected field scan observations (Tello et al. 2000), we have nailed down the level of ground contamination, which so far has demonstrated to be one of the most significant sources of contamination in survey experiments. The collimated feed pattern has a HPBW of $5.4\deg$.

The radiometric signal is sampled at nearly half-second intervals by a thermally controlled total power receiver within a 90 bandwidth. The system temperature is $127.42\pm 0.36$`<!-- -->`{=html} and its gain is $61.010\pm
0.047$`<!-- -->`{=html}. We have also performed a detailed study of the receiver systematics in order to predict its response to changes in the thermal bath of its components, such that the receiver baseline drifts at $0.3591\pm 0.0007$`<!-- -->`{=html} and the relative gain change has a thermal susceptability of $0.01030\pm
0.00001/\deg$C.

# Data Analysis and Results

For some 20% of the data collected at 1465 the stability of the receiver guarantees a 95% confidence level of spurious signal rejection due to gain variations on a timescale of 150-minute long observing runs. By keeping the Sun and the Moon at angles larger than $30\deg$ and $6\deg$, respectively, from axis, a total of 8.5% of data had to be purged; while only 1.6% had to be excised due to RFI signals. In addition, the firings from a stable noise source diode at 44.8-s intervals were used to correct the received signals. Finally, we applied a de-striping algorithm (Tello 1997) to remove low frequency 1/f noise from the remaining observations. The technique relies on a sampling of the circular scans into networks of 120 half-a-beamwidth wide bins which, when divided into Eastern and Western sets, become fiducial baselines to calibrate the Western and Eastern scans that intercept them (see Fig.2). These calibrated scans can in turn be used to compose further bin networks to the East and West in order to propagate the map baseline across an entire declination band.

This technique presumes, however, the absence of stray radiation from the Galactic Plane. We have not yet compensated for this source of systematic effect in the data, which implies in a colder baseline than it should after it has propagated from a warm into a cold region of the sky, but the present status of our work can be best appreciated in the map of Fig. 3. This map combines 63.5 hours of data from the Bishop site with 212.65 hours from the Brazilian site and has been set to a pixel resolution of $2.8125\deg$ spanning an antenna temperature range from $2.45$`<!-- -->`{=html} to $9.65$`<!-- -->`{=html} in $0.15$`<!-- -->`{=html} steps.

The GEM project, whose principal investigators appear as co-authors in the present contribution, has been the work of an outstanding team of researchers; mainly M. Bensadoun, M. Limon, G. De Amici, J. Gibson, J. Yamada, C. Witebsky, D. Heine, M. Leung, A. Alves, L. Arantes, C.A. Wuensche, N. Figueiredo, R. de Souza, A.P. da Silva and, last but not least, J. Aymon. This work was supported by FAPESP through grant No. 00/05436-1.


abstract: We have simulated the optical properties of micro-fabricated Fresnel zone plates (FZPs) as an alternative to spatial light modulators (SLMs) for producing non-trivial light potentials to trap atoms within a lensless Fresnel arrangement. We show that binary (1-bit) FZPs with wavelength ($1\,\mu$m) spatial resolution consistently outperform kinoforms of spatial and phase resolution comparable to commercial SLMs in root mean square error comparisons, with FZP kinoforms demonstrating increasing improvement for complex target intensity distributions. Moreover, as sub-wavelength resolution microfabrication is possible, FZPs provide an exciting possibility for the creation of static cold-atom trapping potentials useful to atomtronics, interferometry, and the study of fundamental physics.
address: Dept. of Physics, SUPA, University of Strathclyde, Glasgow, G4 0NG, UK
author: V A Henderson, P F Griffin, E Riis and A S Arnold$^{1}$
title: Comparative simulations of Fresnel holography methods for atomic waveguides

# Introduction

Atom interferometry is a powerful tool for precise measurements and metrological technologies. It can be used for a wide range of applications, from the determination of fundamental constants and cosmological phenomena to navigation applications such as accelerometers and gyroscopes . Developments in laser cooling, trapping and atom manipulation have allowed a wide range of atom interferometers to be developed , and for the exploration of light based atom traps . Optical traps can offer a method of production for much more complex micrometer scale traps such as atomtronic optical circuits .

Toroidal trapping of cold atoms for use as atom circuits has many applications beyond interferometry , such as the study of persistent currents in superfluids , and low-dimensional atomic systems . However, trapping ultra-cold atoms requires a very smooth trap, as the presence of very small perturbations in a potential can result in heating of a cold atom cloud or fragmentation of a trapped Bose-Einstein condensate . Within previous demonstrations of all-optical ring trapped BECs, the azimuthal variation of the ring minimum was far below the chemical potential of the BEC, with these rings produced through a variety of methods such as painted potentials or combinations of confining light sheets with shaped light, for instance, Laguerre-Gaussian beams , co-axial focused beams , or conical refraction based beams . To successfully produce trapping potentials for BEC, we must aim to match or surpass the above limit on azimuthal variation, thus aiming to produce traps of $\mu$K depth with a roughness of below 1%.

There are many methods which can be used to produce tailored optical potentials, ranging from acousto-optic beam deflection techniques to holographic phase manipulation using a phase adjustable spatial light modulator (SLM) or digital micromirror device (DMD) . To date, the holographic method has proved to be very adaptable, paving the way for the production of novel optical lattices for quantum simulation , dark spontaneous-force optical traps and exotic Laguerre-Gauss modes . Despite these successes, SLM holography for atom trapping still remains an imperfect and computationally intensive technique, notwithstanding significant improvement in the iterative algorithms used . This is due to a combination of system aberrations, low spatial resolution, dead space between pixels, and the difficulty of creating an algorithm that converges on a solution suitable for atom trapping (i.e. smooth and without background light which could cause low loading rates or tunnelling out of the trap ) without lowering light usage efficiency.

Fresnel Zone Plates (FZPs) work by spatially modulating either the amplitude or phase of a light beam, resulting in interference of the optical field after propagation; by design of the modulated region one can in principle then produce an arbitrary optical pattern, or trapping potential for atomtronics. The prototypical FZP is one that acts as a lens, resulting in a focused spot in the selected focal plane ($z = f$). While the operation of such an FZP is standard in the teaching literature of diffraction, we find it intuitive to briefly consider the FZP required to generate a single focus, shown diagrammatically in figure  a). We make use of the time/direction symmetry of linear optics by starting from the desired result and finding the full electric field pattern at a defined plane. Our goal is now to create an optical element, the FZP, that matches an input beam, for example an idealised plane wave, to the field pattern that we produced in the plane. The FZP can then be considered the hologram generated by a plane wave and the backward-propagating field from the focus. For a binary FZP, we obtain a two–level map of the phases of the electric field in the plane of the FZP required to generate the desired focus. In the next section we discuss in detail the theory and numerical methods to implement this. This type of plate (Fig. ) consists of alternating Fresnel zones forming concentric rings that alternate between the chosen binary states at radii, $$r_j = \sqrt{j\lambda f + \frac{j^2 \lambda^2}{4}}~,
\label{eq:fres_r}$$ where $j$ can take any integer value, and $\lambda$ is the wavelength of the incident light. Successive rings can be blocked, allowing only those that constructively interfere at the target plane to propagate. Alternatively, a phase shift of $\pi$ can be added to otherwise 'destructive' zones, increasing the useful power at the focal plane. Figures b), c) demonstrate an envisaged transmissive binary FZP etched into a substrate, with consecutive zones that would be completely out of phase experiencing an increased optical path length.

A similar approach can be used to make straight waveguides with a linear symmetric FZP pattern, or to create arbitrary FZP-like patterns by recording the phase of a near-field diffraction pattern. In this work we will calculate and simulate phase plate patterns, or kinoforms, for single focii, rings, and beamsplitters, as shown in figure . These target intensity distribution have been chosen due to their applicability to cold-atom trapping and atomtronics. The single focus allows both the calculation and propagation methods to be evaluated and compared to the simplest FZP model, whereas the ring allows for comparison of this method to existing toroidal traps which are the simplest nontrivial closed-loop circuits. In order to extend the simulations to consider complex elements for atom optics we finally consider a beam splitter, as such an element is essential as a building block to create a circuit type interferometer.

We anticipate that microfabricated FZPs will overcome many of the limitations posed by the use of SLMs in atom trapping experiments. The higher spatial resolution and sharper edges between pixels presents the ability to reach higher spatial frequency and thus produce a wider range of more accurate holograms. Additionally, due to their size and transmissive operation, we expect that FZPs should be placed inside a vacuum chamber (as with the grating MOTs shown in ), thus immediately addressing the major system aberration of propagation through a vacuum chamber window. Further information will be discussed pertaining to the nature of FZPs in future sections.

# Simulation methods

The phase patterns required to produce the optical traps shown in figure are calculated using a Fourier–optics method of modelling the propagation of an initial electric field $\mathcal{E}^{(0)} = E(x', y', z=0)$ to a distance $z$. This uses the angular spectrum of the field, ($\mathcal{A}^{(0)}$), and the Helmholtz propagator, $\mathcal{H}$, such that, $$\mathcal{E}^{(z)} = \mathcal{F}^{-1}\left [\mathcal{H}(z)  \mathcal{A}^{(0)} \right ] = \mathcal{F}^{-1}\left [e^{ik_z z}  \mathcal{F}\left [ \mathcal{E}^{(0)} \right ] \right ]~,
\label{eq:fourier_prop}$$ where the z-component of the wave vector is $k_z = \sqrt{k^2 - k_x^2 - k_y^2}$ for an electric field with wave vector $k = 2 \pi / \lambda$ . We use this method, following the details in , and references therein, to complete the design algorithm shown in figure ; firstly, a target intensity is calculated and then propagated backwards, using equation , by the focal length. The phase of the resulting electric field in this plane is rounded to the desired bit depth, as discussed later in the text. This routine acts to calculate the required kinoform, and the performance of the result is tested numerically by simulating a desired input beam (either a plane wave or a Gaussian beam with defined width) that is then propagated forward by the focal length. Our method of simulation means that the pixel sizes of the kinoform and simulation (the electric field) are independent. Although we set the input beam and target plane to have flat phase fronts, we allow for phase freedom in the resultant distribution. As we are not utilising a feedback algorithm, our method intrinsically avoids the presence of optical vortices, which can be confirmed through observations of simulation results. We consider the case in which the kinoform acts as a transmissive element and the incident light only illuminates the patterned area. It should also be noted that no optimisation is used to improve the kinoform. This full Helmholtz propagation method is computationally efficient and accurate, reducing the possibility of fringing artifacts in comparison to the paraxial approximation utilised in many hologram calculations as also highlighted in Ref. .

To evaluate the success of each kinoform, we calculate the root mean squared (RMS) error for the normalised two dimensional final and target intensities, $$\epsilon = \sqrt{\frac{1}{N} \sum \left ( \tilde{I}- \tilde{T}\right )^2}~,
\label{eq:rmserror}$$ where $N$ is the number of pixels (in the simulation), $\tilde{I}$ is the final intensity, and $\tilde{T}$ is the target intensity distribution, both intensity distributions are normalised by the mean of the pixels in $T$ that are brighter than $50\%$ of the maximum value .

The target distributions we have chosen to simulate are shown in figure : a) a focus with Gaussian waist (e$^{-2}$ radius) $w_0=5~\mu$m; b) a ring of radius $r=200~\mu$m and radial Gaussian waist $w_r=2~\mu$m; c) a beam splitter formed from straight segments and radii as given in b), again with waist $w_b=2~\mu$m.

Laser parameters of 2 mW (30 mW) power at a wavelength of 1064 nm were used for the focus (ring and beam splitter) simulations as these parameters give trap depths of a few $\mu$K. Moreover, trap frequencies are 2 kHz in the direction of tightest confinement, which is higher than existing ring shaped dipole potentials and permits access to lower dimensional regimes. The ring radius is larger than these previous demonstrations to increase its applicability to interferometry application where sensitivity scales with the area enclosed.

Within the simulations, we run the calculations for a wide range of kinoform pixel sizes and phase resolution (or bit depth), allowing the comparison of binary FZP-type kinoforms with simulated pixel size of 1 $\mu$m to 8-bit SLM type kinoforms with simulated pixel sizes of 12 $\mu$m or more. The 12 $\mu$m pixel size corresponds to the state of the art for SLMs, which have an effective area of approximately 2 cm$^2$, whereas FZPs can be manufactured with pixel size as small as 10 nm and with large total areas of up to 25 cm$^2$ . Despite these evident spatial advantages for FZPs, one must remember that SLMs typically operate with 8-bit precision and are updatable, whereas FZPs, by their very nature are static, with only two levels of phase control. Both technologies are already being utilised for trapping, in the form of optical tweezers .

Throughout the simulation process, the electric field propagation is calculated to a resolution of a wavelength with a simulation area of 4.38$\times$`<!-- -->`{=html}4.38 mm$^2$ ($2^{12}\,\lambda\times2^{12}\,\lambda$), limited solely by the reverse propagation technique and computation memory requirements. For illumination by Gaussian beams, the choice of the input beam e$^{-2}$ radius, $w(z)$, is determined by the desired focal length and the Gaussian width, $w_0$ of the desired features by $w(z)=w_0\sqrt{1+(z/z_{\rm R})^2}$, with Rayleigh length $z_{\rm R}=\pi\,w_0^2/\lambda$. We do note that these computation limitations mean that the active area is smaller than, if comparable to, typical SLM active areas.

# Results & Discussion

Maps of RMS error, calculated in the simulations using equation , are shown in figures and . For all three target patterns and illumination beams (except the plane focus), there is a clear increase in RMS error with increasing pixel size and decreasing bit depth. The simulations also show that a two level FZP consistently has an RMS error lower than that of a kinoform comparable to an SLM. In addition, we can note that, at low pixel size, increasing the bit depth from 2 to 4 level phase resolution significantly reduces the RMS error, thus improvements in microfabrication techniques would significantly increase the accuracy of the FZP kinoforms by allowing for non-binary phase. Examples of the calculated kinoforms for FZPs illuminated by a Gaussian and producing a ring and beam splitter are shown in figure .

The RMS error map, shown in figure , for a focus kinoform illuminated by a plane wave clearly shows an unexpected increase in RMS error at high phase (high bit depth) and spatial resolution (small pixel size). In this area of higher RMS error, we observe that the optical power is concentrated in a tighter focus than the target 5 $\mu$m e$^{-2}$ radius. This can be understood because each pixel of the kinoform is illuminated equally, unlike Gaussian optics where a concomitant Gaussian illumination of the optical element is required. We can explore the consequences of this by considering three of the contributors to the RMS error: phase resolution error, spatial resolution error, and illumination error. In our algorithm all spatial intensity information from back-propagation is lost and replaced by the intensity information of the illumination beam, whereas the phase information loss is only limited by the pixel size and phase resolution. At large pixel size and low phase resolution, these sources of error dominate over the intensity error, but at high resolutions, the lost intensity information becomes dominant. It is a standard result in Gaussian optics that a smaller focus diverges more rapidly than a larger focus, meaning that the tighter the focus desired, the larger a kinoform or lens should be used, such that the numerical aperture can be increased. Conversely, this means that the size of the illuminated area of the kinoform, rather than the phase across it, affects the size of the focus produced. So, for the plane wave case, the illumination is more similar to that required for a smaller focus than 5 $\mu$m. We do not see this in the Gaussian illumination simulations, figure , due to the Gaussian weighting of the intensity at the kinoform.

As one can see from the RMS errors shown in figures and , accuracy of intensity reproduction is reduced with pattern complexity and for distributions with less obvious symmetries: reproduction of the beam splitter is much less accurate than for either the focus and the ring. Both the ring and the focus have been masked to form circularly symmetric kinoforms, meaning that artefacts caused by the square shape of the active area are reduced, however, the reduced symmetry of the beam splitter makes this process more complex. The masking makes pixels outside of a desired area completely dark, thus creating an 'active area' of illuminated pixels and excluding pixels which cause abberations. In the beam-splitter case, we were able to use the symmetry properties of a straight waveguide Fresnel zone plate to shape the active area appropriately, thus blocking light incident more than a certain distance from centre of the intensity lines. This technique greatly improved accuracy, but requires further fine-tuning to allow the approach to be applied to an arbitrary intensity pattern. However, we note that if the appropriate spatial distribution of the incident field, with a flat phase front, can be produced at the kinoform, then the errors would rapidly tend to zero, as for the single focus in the upper plots of figure . Indeed, producing such a large scale pattern is well suited to the coarser resolution of an SLM, suggesting that SLMs and FZPs can be used together synergistically.

In all the error maps, particularly for the Gaussian illumination, we see non-monotonic variations in the errors between consecutive pixel sizes. This is due to aliasing between the three length scales involved in the kinoform design calculations: the length scale of phase change, the simulation pixel size ($\lambda$), and the kinoform pixel size. Due to the involvement of three length scales we were not able to reduce this roughness with suitable choice of any of these values. The roughness in RMS error is less pronounced for plane wave illumination as the overall RMS error is higher and so this aliasing is less prominent.

We can also note that the discontinuous nature of the example beam splitter has also increased the error in its production, this led to us using a target that reached the edges of the simulation area to avoid such issues. In a useful intensity distribution for atomtronics, one would want to produce a target intensity with no discontinuities (i.e. a closed-loop circuit), such as a ring with a beam splitter at either end for use in interferometry; hence, the discontinuity based artifacts and errors are not critical to the success of these simulations.

In order to demonstrate the applications of the hologram method of optical trap generation (particularly the potential for three dimensional trapping), we have demonstrated propagation through the focus of the ring distribution in figure . This is shown both for the best kinoform and for an FZP, with the average intensity of the ring minimum at each distance shown as a scatter plot alongside the full intensity distributions. Both cases demonstrate a full-width-half-maximum (FWHM) in the propagation direction of 20 $\mu$m, similar to that expected for a focussed Gaussian beam. Azimuthal plots of intensity are shown as line graphs in figure , allowing for intensity noise to be seen. We can note that the intensity distribution in the case of the focus and ring are too narrow to show any noise due to the pixel size of the simulation, however, we can see significant noise along the vertical waveguide section of the beam splitter. The beam splitter is noise is largely due to beating between the vertical and horizontal sections of the waveguides and could be minimised with more careful target distribution design.

In the simulations of RMS errors in Figures , we adopted a compromise position whereby we compared both target and image distributions across the whole grid size. This means that even the background wings (i.e. non-target zone) of the intensity distribution - which could affect the atomtronic circuit loading efficiency - contribute to the error. However, for a given application one may be mainly interested in a subset of the image and target, e.g. the pixel region where the top 50% of the target intensity distribution. This region is where the coldest atoms would be trapped and in this case it makes sense to modify equation to only consider pixels in this zone. Moreover, one should then adapt $\tilde{I}$ the final intensity, and $\tilde{T}$ the target intensity distribution, so that the intensity distributions are independently normalised by their maximum value over the pixels in $T$ which are brighter than $50\%$ of the maximum value. This gives a more realistic estimate of the in-situ trap roughness, which can be seen in figure . The lowest RMS error, typical FZP, and typical SLM have corresponding errors of $0.0\%,$ $3.7\,\%$ and $32.7\%,$ respectively for a ring shaped target. In this situation, rather than the plane wave/Gaussian illumination considered in Figs.  and , the hologram is illuminated by its ideal spatial intensity distribution – a realistic assumption we elucidate on in our conclusions.

# Outlook and Conclusion

By calculating and simulating kinoforms for focii, rings and beam splitters, we have shown that under most circumstances spatial resolution is much more critical than the bit-depth of the hologram. Specifically, we demonstrated that, in the lensless Fresnel regime, FZPs with wavelength spatial resolution consistently show improved root mean square error over kinoforms of spatial and phase resolution comparable to commercial SLMs which are typically 8—bit, with 12 $\mu$m pixels. FZP kinoforms become increasingly superior for complex target intensity distributions, indicating their suitability for use to produce static atomtronic circuits for trapping ultracold atoms. This is accompanied by the illustration of 3D trapping capabilities through propagation of a ring shaped potential through its focus. By extension of a FZP from a binary kinoform to a 4 level kinoform, the fidelity of intensity distributions can be greatly increased, showing the potential of these kinoforms to improve with increasing micro-fabrication capabilities .

Despite the success of these simulations, they are limited to wavelength resolution due to the Helmholtz propagation method used and to a size of 4.38$\times$`<!-- -->`{=html}4.38 mm$^2$ by the memory requirements of the simulation. The calculation process explicitly does not include an algorithm for iterative error correction of the kinoform meaning that both FZP and SLM RMS errors may find improvements with the use of algorithms similar to those used in .

Future work will extend this method of kinoform calculation to include an optimisation algorithm, whilst the manufacture of potentially useful FZPs will allow for predictions to be tested experimentally. There is also great potential for combining the strengths of different techniques: a laser incident on a DMD pattern could be re-imaged onto the FZP in order to provide flat-phase front spatially-tailored intensity illumination (assumed in Fig. ), with RMS errors substantially reduced beyond those from plane wave/Gaussian illumination. Whilst only red-detuned (bright) dipole potentials were considered in this paper, extension to blue-detuned (dark) traps should be straightforward, however such patterns rely more heavily on destructive interference which is likely to impinge on the smoothness of the final patterns. Additionally, FZPs should lend themselves to future extension work involving multi-wavelength hologram production following a similar approach to that shown in .

For the dataset associated with this paper please see .

# Acknowledgments

This work has been supported by DSTL, the Leverhulme Trust RPG-2013-074 and by the EPSRC as part of the Quantum Technology Hub for Sensors and Metrology. We are grateful to Jonathan Pritchard for initial assistance with the simulations.

# References