title: How to get the weekday from day of month using pyspark
body: <p>I have a dataframe log_df:
  <a href="https://i.stack.imgur.com/IeFg1.jpg" rel="noreferrer"><img src="https://i.stack.imgur.com/IeFg1.jpg" alt="enter image description here"></a></p>
  
  <p>I generate a new dataframe based on the following code:</p>
  
  <pre><code>from pyspark.sql.functions import split, regexp_extract 
  split_log_df = log_df.select(regexp_extract('value', r'^([^\s]+\s)', 1).alias('host'),
                            regexp_extract('value', r'^.*\[(\d\d/\w{3}/\d{4}:\d{2}:\d{2}:\d{2} -\d{4})]', 1).alias('timestamp'),
                            regexp_extract('value', r'^.*"\w+\s+([^\s]+)\s+HTTP.*"', 1).alias('path'),
                            regexp_extract('value', r'^.*"\s+([^\s]+)', 1).cast('integer').alias('status'),
                            regexp_extract('value', r'^.*\s+(\d+)$', 1).cast('integer').alias('content_size'))
  split_log_df.show(10, truncate=False)
  </code></pre>
  
  <p>the new dataframe is like:
  <a href="https://i.stack.imgur.com/DQT0t.jpg" rel="noreferrer"><img src="https://i.stack.imgur.com/DQT0t.jpg" alt="enter image description here"></a></p>
  
  <p>I need another column showing the dayofweek, what would be the best elegant way to create it? ideally just adding a udf like field in the select.</p>
  
  <p>Thank you very much.</p>
  
  <p>Updated: my question is different than the one in the comment, what I need is to make the calculation based on a string in log_df, not based on the timestamp like the comment, so this is not a duplicate question. Thanks.</p>