pyspark.sql.functions.to_unix_timestamp#

pyspark.sql.functions.to_unix_timestamp(timestamp, format=None)[source]#

Returns the UNIX timestamp of the given time.

New in version 3.5.0.

Parameters

timestampColumn or str: Input column or strings.
formatColumn or str, optional: format to use to convert UNIX timestamp values.

Examples

>>> spark.conf.set("spark.sql.session.timeZone", "America/Los_Angeles")

Example 1: Using default format ‘yyyy-MM-dd HH:mm:ss’ parses the timestamp string.

>>> import pyspark.sql.functions as sf
>>> time_df = spark.createDataFrame([('2015-04-08 12:12:12',)], ['dt'])
>>> time_df.select(sf.to_unix_timestamp('dt').alias('unix_time')).show()
+----------+
| unix_time|
+----------+
|1428520332|
+----------+

Example 2: Using user-specified format ‘yyyy-MM-dd’ parses the timestamp string.

>>> import pyspark.sql.functions as sf
>>> time_df = spark.createDataFrame([('2015-04-08',)], ['dt'])
>>> time_df.select(
...     sf.to_unix_timestamp('dt', sf.lit('yyyy-MM-dd')).alias('unix_time')).show()
+----------+
| unix_time|
+----------+
|1428476400|
+----------+

>>> spark.conf.unset("spark.sql.session.timeZone")