awacke1 commited on
Commit
120f67f
·
verified ·
1 Parent(s): 1799381

Update app.py

Browse files
Files changed (1) hide show
  1. app.py +162 -1
app.py CHANGED
@@ -2096,7 +2096,168 @@ st.sidebar.markdown("""
2096
  27 Jun 2023 | Evaluation of OpenAI Codex for HPC Parallel Programming Models Kernel Generation | ⬇️
2097
  William F. Godoy, Pedro Valero-Lara, Keita Teranishi, Prasanna Balaprakash, Jeffrey S. Vetter
2098
 
2099
- We evaluate AI-assisted generative capabilities on fundamental numerical kernels in high-performance computing (HPC), including AXPY, GEMV, GEMM, SpMV, Jacobi Stencil, and CG. We test the generated kernel codes for a variety of language-supported programming models, including (1) C++ (e.g., OpenMP [including offload], OpenACC, Kokkos, SyCL, CUDA, and HIP), (2) Fortran (e.g., OpenMP [including offload] and OpenACC), (3) Python (e.g., numba, Numba, cuPy, and pyCUDA), and (4) Julia (e.g., Threads, CUDA.jl, AMDGPU.jl, and KernelAbstractions.jl). We use the GitHub Copilot capabilities powered by OpenAI Codex available in Visual Studio Code as of April 2023 to generate a vast amount of implementations given simple <kernel> + <programming model> + <optional hints> prompt variants. To quantify and compare the results, we propose a proficiency metric around the initial 10 suggestions given for each prompt. Results suggest that the OpenAI Codex outputs for C++ correlate with the adoption and maturity of programming models. For example, OpenMP and CUDA score really high, whereas HIP is still lacking. We found that prompts from either a targeted language such as Fortran or the more general-purpose Python can benefit from adding code keywords, while Julia prompts perform acceptably well for its mature programming models (e.g., Threads and CUDA.jl). We expect for these benchmarks to provide a point of reference for each programming model's community. Overall, understanding the convergence of large language models, AI, and HPC is crucial due to its rapidly evolving nature and how it is redefining human-computer interactions.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2100
  """)
2101
 
2102
 
 
2096
  27 Jun 2023 | Evaluation of OpenAI Codex for HPC Parallel Programming Models Kernel Generation | ⬇️
2097
  William F. Godoy, Pedro Valero-Lara, Keita Teranishi, Prasanna Balaprakash, Jeffrey S. Vetter
2098
 
2099
+ Date: 27 Jun 2023
2100
+
2101
+ Title: Evaluation of OpenAI Codex for HPC Parallel Programming Models Kernel Generation
2102
+
2103
+ Abstract Link: https://arxiv.org/abs/2306.15121
2104
+
2105
+ PDF Link: https://arxiv.org/pdf/2306.15121
2106
+
2107
+ Date: 27 Mar 2020
2108
+
2109
+ Title: Detecting and Characterizing Bots that Commit Code
2110
+
2111
+ Abstract Link: https://arxiv.org/abs/2003.03172
2112
+
2113
+ PDF Link: https://arxiv.org/pdf/2003.03172
2114
+
2115
+ Date: 05 Feb 2022
2116
+
2117
+ Title: High-performance symbolic-numerics via multiple dispatch
2118
+
2119
+ Abstract Link: https://arxiv.org/abs/2105.03949
2120
+
2121
+ PDF Link: https://arxiv.org/pdf/2105.03949
2122
+
2123
+ Date: 12 Apr 2022
2124
+
2125
+ Title: Learning to Represent Programs with Heterogeneous Graphs
2126
+
2127
+ Abstract Link: https://arxiv.org/abs/2012.04188
2128
+
2129
+ PDF Link: https://arxiv.org/pdf/2012.04188
2130
+
2131
+ Date: 11 Jul 2023
2132
+
2133
+ Title: DyCL: Dynamic Neural Network Compilation Via Program Rewriting and Graph Optimization
2134
+
2135
+ Abstract Link: https://arxiv.org/abs/2307.04963
2136
+
2137
+ PDF Link: https://arxiv.org/pdf/2307.04963
2138
+
2139
+ Date: 21 Jan 2021
2140
+
2141
+ Title: PyGlove: Symbolic Programming for Automated Machine Learning
2142
+
2143
+ Abstract Link: https://arxiv.org/abs/2101.08809
2144
+
2145
+ PDF Link: https://arxiv.org/pdf/2101.08809
2146
+
2147
+ Date: 26 Nov 2022
2148
+
2149
+ Title: Sketch2FullStack: Generating Skeleton Code of Full Stack Website and Application from Sketch using Deep Learning and Computer Vision
2150
+
2151
+ Abstract Link: https://arxiv.org/abs/2211.14607
2152
+
2153
+ PDF Link: https://arxiv.org/pdf/2211.14607
2154
+
2155
+ Date: 25 May 2023
2156
+
2157
+ Title: Code as Policies: Language Model Programs for Embodied Control
2158
+
2159
+ Abstract Link: https://arxiv.org/abs/2209.07753
2160
+
2161
+ PDF Link: https://arxiv.org/pdf/2209.07753
2162
+
2163
+ Date: 30 Nov 2023
2164
+
2165
+ Title: CADTalk: An Algorithm and Benchmark for Semantic Commenting of CAD Programs
2166
+
2167
+ Abstract Link: https://arxiv.org/abs/2311.16703
2168
+
2169
+ PDF Link: https://arxiv.org/pdf/2311.16703
2170
+
2171
+ Date: 27 Jan 2020
2172
+
2173
+ Title: CodeReef: an open platform for portable MLOps, reusable automation actions and reproducible benchmarking
2174
+
2175
+ Abstract Link: https://arxiv.org/abs/2001.07935
2176
+
2177
+ PDF Link: https://arxiv.org/pdf/2001.07935
2178
+
2179
+ Date: 10 Jul 2023
2180
+
2181
+ Title: COMEX: A Tool for Generating Customized Source Code Representations
2182
+
2183
+ Abstract Link: https://arxiv.org/abs/2307.04693
2184
+
2185
+ PDF Link: https://arxiv.org/pdf/2307.04693
2186
+
2187
+ Date: 10 Jul 2023
2188
+
2189
+ Title: AmadeusGPT: a natural language interface for interactive animal behavioral analysis
2190
+
2191
+ Abstract Link: https://arxiv.org/abs/2307.04858
2192
+
2193
+ PDF Link: https://arxiv.org/pdf/2307.04858
2194
+
2195
+ Date: 19 Mar 2021
2196
+
2197
+ Title: API2Com: On the Improvement of Automatically Generated Code Comments Using API Documentations
2198
+
2199
+ Abstract Link: https://arxiv.org/abs/2103.10668
2200
+
2201
+ PDF Link: https://arxiv.org/pdf/2103.10668
2202
+
2203
+ Date: 29 Oct 2020
2204
+
2205
+ Title: IntelliCode Compose: Code Generation Using Transformer
2206
+
2207
+ Abstract Link: https://arxiv.org/abs/2005.08025
2208
+
2209
+ PDF Link: https://arxiv.org/pdf/2005.08025
2210
+
2211
+ Date: 22 Mar 2022
2212
+
2213
+ Title: Demo of the Linguistic Field Data Management and Analysis System -- LiFE
2214
+
2215
+ Abstract Link: https://arxiv.org/abs/2203.11443
2216
+
2217
+ PDF Link: https://arxiv.org/pdf/2203.11443
2218
+
2219
+ Date: 30 Jan 2024
2220
+
2221
+ Title: ReactGenie: A Development Framework for Complex Multimodal Interactions Using Large Language Models
2222
+
2223
+ Abstract Link: https://arxiv.org/abs/2306.09649
2224
+
2225
+ PDF Link: https://arxiv.org/pdf/2306.09649
2226
+
2227
+ Date: 09 Feb 2024
2228
+
2229
+ Title: Model Editing with Canonical Examples
2230
+
2231
+ Abstract Link: https://arxiv.org/abs/2402.06155
2232
+
2233
+ PDF Link: https://arxiv.org/pdf/2402.06155
2234
+
2235
+ Date: 11 Jun 2021
2236
+
2237
+ Title: PyGAD: An Intuitive Genetic Algorithm Python Library
2238
+
2239
+ Abstract Link: https://arxiv.org/abs/2106.06158
2240
+
2241
+ PDF Link: https://arxiv.org/pdf/2106.06158
2242
+
2243
+ Date: 27 Nov 2020
2244
+
2245
+ Title: ROS Based Visual Programming Tool for Mobile Robot Education and Applications
2246
+
2247
+ Abstract Link: https://arxiv.org/abs/2011.13706
2248
+
2249
+ PDF Link: https://arxiv.org/pdf/2011.13706
2250
+
2251
+ Date: 28 Sep 2021
2252
+
2253
+ Title: A Toolkit for Generating Code Knowledge Graphs
2254
+
2255
+ Abstract Link: https://arxiv.org/abs/2002.09440
2256
+
2257
+ PDF Link: https://arxiv.org/pdf/2002.09440
2258
+
2259
+
2260
+
2261
  """)
2262
 
2263