{
"docs": [
{
"location": "/",
"text": "!\n\n\n! expr - Logical not.\n\n\n\n\n%\n\n\nexpr1 % expr2 - Returns the remainder after \nexpr1\n/\nexpr2\n.\n\n\nExamples:\n\n\n> SELECT 2 % 1.8;\n 0.2\n> SELECT MOD(2, 1.8);\n 0.2\n\n\n\n\n\n\n&\n\n\nexpr1 & expr2 - Returns the result of bitwise AND of \nexpr1\n and \nexpr2\n.\n\n\nExamples:\n\n\n> SELECT 3 & 5;\n 1\n\n\n\n\n\n\n*\n\n\nexpr1 * expr2 - Returns \nexpr1\n*\nexpr2\n.\n\n\nExamples:\n\n\n> SELECT 2 * 3;\n 6\n\n\n\n\n\n\n+\n\n\nexpr1 + expr2 - Returns \nexpr1\n+\nexpr2\n.\n\n\nExamples:\n\n\n> SELECT 1 + 2;\n 3\n\n\n\n\n\n\n-\n\n\nexpr1 - expr2 - Returns \nexpr1\n-\nexpr2\n.\n\n\nExamples:\n\n\n> SELECT 2 - 1;\n 1\n\n\n\n\n\n\n/\n\n\nexpr1 / expr2 - Returns \nexpr1\n/\nexpr2\n. It always performs floating point division.\n\n\nExamples:\n\n\n> SELECT 3 / 2;\n 1.5\n> SELECT 2L / 2L;\n 1.0\n\n\n\n\n\n\n<\n\n\nexpr1 < expr2 - Returns true if \nexpr1\n is less than \nexpr2\n.\n\n\nArguments:\n\n\n\n\nexpr1, expr2 - the two expressions must be same type or can be casted to a common type,\n and must be a type that can be ordered. For example, map type is not orderable, so it\n is not supported. For complex types such array/struct, the data types of fields must\n be orderable.\n\n\n\n\nExamples:\n\n\n> SELECT 1 < 2;\n true\n> SELECT 1.1 < '1';\n false\n> SELECT to_date('2009-07-30 04:17:52') < to_date('2009-07-30 04:17:52');\n false\n> SELECT to_date('2009-07-30 04:17:52') < to_date('2009-08-01 04:17:52');\n true\n> SELECT 1 < NULL;\n NULL\n\n\n\n\n\n\n<=\n\n\nexpr1 <= expr2 - Returns true if \nexpr1\n is less than or equal to \nexpr2\n.\n\n\nArguments:\n\n\n\n\nexpr1, expr2 - the two expressions must be same type or can be casted to a common type,\n and must be a type that can be ordered. For example, map type is not orderable, so it\n is not supported. For complex types such array/struct, the data types of fields must\n be orderable.\n\n\n\n\nExamples:\n\n\n> SELECT 2 <= 2;\n true\n> SELECT 1.0 <= '1';\n true\n> SELECT to_date('2009-07-30 04:17:52') <= to_date('2009-07-30 04:17:52');\n true\n> SELECT to_date('2009-07-30 04:17:52') <= to_date('2009-08-01 04:17:52');\n true\n> SELECT 1 <= NULL;\n NULL\n\n\n\n\n\n\n<=>\n\n\nexpr1 <=> expr2 - Returns same result as the EQUAL(=) operator for non-null operands,\nbut returns true if both are null, false if one of the them is null.\n\n\nArguments:\n\n\n\n\nexpr1, expr2 - the two expressions must be same type or can be casted to a common type,\n and must be a type that can be used in equality comparison. Map type is not supported.\n For complex types such array/struct, the data types of fields must be orderable.\n\n\n\n\nExamples:\n\n\n> SELECT 2 <=> 2;\n true\n> SELECT 1 <=> '1';\n true\n> SELECT true <=> NULL;\n false\n> SELECT NULL <=> NULL;\n true\n\n\n\n\n\n\n=\n\n\nexpr1 = expr2 - Returns true if \nexpr1\n equals \nexpr2\n, or false otherwise.\n\n\nArguments:\n\n\n\n\nexpr1, expr2 - the two expressions must be same type or can be casted to a common type,\n and must be a type that can be used in equality comparison. Map type is not supported.\n For complex types such array/struct, the data types of fields must be orderable.\n\n\n\n\nExamples:\n\n\n> SELECT 2 = 2;\n true\n> SELECT 1 = '1';\n true\n> SELECT true = NULL;\n NULL\n> SELECT NULL = NULL;\n NULL\n\n\n\n\n\n\n==\n\n\nexpr1 == expr2 - Returns true if \nexpr1\n equals \nexpr2\n, or false otherwise.\n\n\nArguments:\n\n\n\n\nexpr1, expr2 - the two expressions must be same type or can be casted to a common type,\n and must be a type that can be used in equality comparison. Map type is not supported.\n For complex types such array/struct, the data types of fields must be orderable.\n\n\n\n\nExamples:\n\n\n> SELECT 2 == 2;\n true\n> SELECT 1 == '1';\n true\n> SELECT true == NULL;\n NULL\n> SELECT NULL == NULL;\n NULL\n\n\n\n\n\n\n>\n\n\nexpr1 > expr2 - Returns true if \nexpr1\n is greater than \nexpr2\n.\n\n\nArguments:\n\n\n\n\nexpr1, expr2 - the two expressions must be same type or can be casted to a common type,\n and must be a type that can be ordered. For example, map type is not orderable, so it\n is not supported. For complex types such array/struct, the data types of fields must\n be orderable.\n\n\n\n\nExamples:\n\n\n> SELECT 2 > 1;\n true\n> SELECT 2 > '1.1';\n true\n> SELECT to_date('2009-07-30 04:17:52') > to_date('2009-07-30 04:17:52');\n false\n> SELECT to_date('2009-07-30 04:17:52') > to_date('2009-08-01 04:17:52');\n false\n> SELECT 1 > NULL;\n NULL\n\n\n\n\n\n\n>=\n\n\nexpr1 >= expr2 - Returns true if \nexpr1\n is greater than or equal to \nexpr2\n.\n\n\nArguments:\n\n\n\n\nexpr1, expr2 - the two expressions must be same type or can be casted to a common type,\n and must be a type that can be ordered. For example, map type is not orderable, so it\n is not supported. For complex types such array/struct, the data types of fields must\n be orderable.\n\n\n\n\nExamples:\n\n\n> SELECT 2 >= 1;\n true\n> SELECT 2.0 >= '2.1';\n false\n> SELECT to_date('2009-07-30 04:17:52') >= to_date('2009-07-30 04:17:52');\n true\n> SELECT to_date('2009-07-30 04:17:52') >= to_date('2009-08-01 04:17:52');\n false\n> SELECT 1 >= NULL;\n NULL\n\n\n\n\n\n\n^\n\n\nexpr1 ^ expr2 - Returns the result of bitwise exclusive OR of \nexpr1\n and \nexpr2\n.\n\n\nExamples:\n\n\n> SELECT 3 ^ 5;\n 6\n\n\n\n\n\n\nabs\n\n\nabs(expr) - Returns the absolute value of the numeric value.\n\n\nExamples:\n\n\n> SELECT abs(-1);\n 1\n\n\n\n\n\n\nacos\n\n\nacos(expr) - Returns the inverse cosine (a.k.a. arc cosine) of \nexpr\n, as if computed by\n\njava.lang.Math.acos\n.\n\n\nExamples:\n\n\n> SELECT acos(1);\n 0.0\n> SELECT acos(2);\n NaN\n\n\n\n\n\n\nadd_months\n\n\nadd_months(start_date, num_months) - Returns the date that is \nnum_months\n after \nstart_date\n.\n\n\nExamples:\n\n\n> SELECT add_months('2016-08-31', 1);\n 2016-09-30\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nand\n\n\nexpr1 and expr2 - Logical AND.\n\n\n\n\napprox_count_distinct\n\n\napprox_count_distinct(expr[, relativeSD]) - Returns the estimated cardinality by HyperLogLog++.\n\nrelativeSD\n defines the maximum estimation error allowed.\n\n\n\n\napprox_percentile\n\n\napprox_percentile(col, percentage [, accuracy]) - Returns the approximate percentile value of numeric\ncolumn \ncol\n at the given percentage. The value of percentage must be between 0.0\nand 1.0. The \naccuracy\n parameter (default: 10000) is a positive numeric literal which\ncontrols approximation accuracy at the cost of memory. Higher value of \naccuracy\n yields\nbetter accuracy, \n1.0/accuracy\n is the relative error of the approximation.\nWhen \npercentage\n is an array, each value of the percentage array must be between 0.0 and 1.0.\nIn this case, returns the approximate percentile array of column \ncol\n at the given\npercentage array.\n\n\nExamples:\n\n\n> SELECT approx_percentile(10.0, array(0.5, 0.4, 0.1), 100);\n [10.0,10.0,10.0]\n> SELECT approx_percentile(10.0, 0.5, 100);\n 10.0\n\n\n\n\n\n\narray\n\n\narray(expr, ...) - Returns an array with the given elements.\n\n\nExamples:\n\n\n> SELECT array(1, 2, 3);\n [1,2,3]\n\n\n\n\n\n\narray_contains\n\n\narray_contains(array, value) - Returns true if the array contains the value.\n\n\nExamples:\n\n\n> SELECT array_contains(array(1, 2, 3), 2);\n true\n\n\n\n\n\n\nascii\n\n\nascii(str) - Returns the numeric value of the first character of \nstr\n.\n\n\nExamples:\n\n\n> SELECT ascii('222');\n 50\n> SELECT ascii(2);\n 50\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nasin\n\n\nasin(expr) - Returns the inverse sine (a.k.a. arc sine) the arc sin of \nexpr\n,\nas if computed by \njava.lang.Math.asin\n.\n\n\nExamples:\n\n\n> SELECT asin(0);\n 0.0\n> SELECT asin(2);\n NaN\n\n\n\n\n\n\nassert_true\n\n\nassert_true(expr) - Throws an exception if \nexpr\n is not true.\n\n\nExamples:\n\n\n> SELECT assert_true(0 < 1);\n NULL\n\n\n\n\n\n\natan\n\n\natan(expr) - Returns the inverse tangent (a.k.a. arc tangent) of \nexpr\n, as if computed by\n\njava.lang.Math.atan\n\n\nExamples:\n\n\n> SELECT atan(0);\n 0.0\n\n\n\n\n\n\natan2\n\n\natan2(exprY, exprX) - Returns the angle in radians between the positive x-axis of a plane\nand the point given by the coordinates (\nexprX\n, \nexprY\n), as if computed by\n\njava.lang.Math.atan2\n.\n\n\nArguments:\n\n\n\n\nexprY - coordinate on y-axis\n\n\nexprX - coordinate on x-axis\n\n\n\n\nExamples:\n\n\n> SELECT atan2(0, 0);\n 0.0\n\n\n\n\n\n\navg\n\n\navg(expr) - Returns the mean calculated from values of a group.\n\n\n\n\nbase64\n\n\nbase64(bin) - Converts the argument from a binary \nbin\n to a base 64 string.\n\n\nExamples:\n\n\n> SELECT base64('Spark SQL');\n U3BhcmsgU1FM\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nbigint\n\n\nbigint(expr) - Casts the value \nexpr\n to the target data type \nbigint\n.\n\n\n\n\nbin\n\n\nbin(expr) - Returns the string representation of the long value \nexpr\n represented in binary.\n\n\nExamples:\n\n\n> SELECT bin(13);\n 1101\n> SELECT bin(-13);\n 1111111111111111111111111111111111111111111111111111111111110011\n> SELECT bin(13.3);\n 1101\n\n\n\n\n\n\nbinary\n\n\nbinary(expr) - Casts the value \nexpr\n to the target data type \nbinary\n.\n\n\n\n\nbit_length\n\n\nbit_length(expr) - Returns the bit length of string data or number of bits of binary data.\n\n\nExamples:\n\n\n> SELECT bit_length('Spark SQL');\n 72\n\n\n\n\nSince:\n 2.3.0\n\n\n\n\nboolean\n\n\nboolean(expr) - Casts the value \nexpr\n to the target data type \nboolean\n.\n\n\n\n\nbround\n\n\nbround(expr, d) - Returns \nexpr\n rounded to \nd\n decimal places using HALF_EVEN rounding mode.\n\n\nExamples:\n\n\n> SELECT bround(2.5, 0);\n 2.0\n\n\n\n\n\n\ncast\n\n\ncast(expr AS type) - Casts the value \nexpr\n to the target data type \ntype\n.\n\n\nExamples:\n\n\n> SELECT cast('10' as int);\n 10\n\n\n\n\n\n\ncbrt\n\n\ncbrt(expr) - Returns the cube root of \nexpr\n.\n\n\nExamples:\n\n\n> SELECT cbrt(27.0);\n 3.0\n\n\n\n\n\n\nceil\n\n\nceil(expr) - Returns the smallest integer not smaller than \nexpr\n.\n\n\nExamples:\n\n\n> SELECT ceil(-0.1);\n 0\n> SELECT ceil(5);\n 5\n\n\n\n\n\n\nceiling\n\n\nceiling(expr) - Returns the smallest integer not smaller than \nexpr\n.\n\n\nExamples:\n\n\n> SELECT ceiling(-0.1);\n 0\n> SELECT ceiling(5);\n 5\n\n\n\n\n\n\nchar\n\n\nchar(expr) - Returns the ASCII character having the binary equivalent to \nexpr\n. If n is larger than 256 the result is equivalent to chr(n % 256)\n\n\nExamples:\n\n\n> SELECT char(65);\n A\n\n\n\n\nSince:\n 2.3.0\n\n\n\n\nchar_length\n\n\nchar_length(expr) - Returns the character length of string data or number of bytes of binary data. The length of string data includes the trailing spaces. The length of binary data includes binary zeros.\n\n\nExamples:\n\n\n> SELECT char_length('Spark SQL ');\n 10\n> SELECT CHAR_LENGTH('Spark SQL ');\n 10\n> SELECT CHARACTER_LENGTH('Spark SQL ');\n 10\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\ncharacter_length\n\n\ncharacter_length(expr) - Returns the character length of string data or number of bytes of binary data. The length of string data includes the trailing spaces. The length of binary data includes binary zeros.\n\n\nExamples:\n\n\n> SELECT character_length('Spark SQL ');\n 10\n> SELECT CHAR_LENGTH('Spark SQL ');\n 10\n> SELECT CHARACTER_LENGTH('Spark SQL ');\n 10\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nchr\n\n\nchr(expr) - Returns the ASCII character having the binary equivalent to \nexpr\n. If n is larger than 256 the result is equivalent to chr(n % 256)\n\n\nExamples:\n\n\n> SELECT chr(65);\n A\n\n\n\n\nSince:\n 2.3.0\n\n\n\n\ncoalesce\n\n\ncoalesce(expr1, expr2, ...) - Returns the first non-null argument if exists. Otherwise, null.\n\n\nExamples:\n\n\n> SELECT coalesce(NULL, 1, NULL);\n 1\n\n\n\n\nSince:\n 1.0.0\n\n\n\n\ncollect_list\n\n\ncollect_list(expr) - Collects and returns a list of non-unique elements.\n\n\n\n\ncollect_set\n\n\ncollect_set(expr) - Collects and returns a set of unique elements.\n\n\n\n\nconcat\n\n\nconcat(str1, str2, ..., strN) - Returns the concatenation of str1, str2, ..., strN.\n\n\nExamples:\n\n\n> SELECT concat('Spark', 'SQL');\n SparkSQL\n\n\n\n\n\n\nconcat_ws\n\n\nconcat_ws(sep, [str | array(str)]+) - Returns the concatenation of the strings separated by \nsep\n.\n\n\nExamples:\n\n\n> SELECT concat_ws(' ', 'Spark', 'SQL');\n Spark SQL\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nconv\n\n\nconv(num, from_base, to_base) - Convert \nnum\n from \nfrom_base\n to \nto_base\n.\n\n\nExamples:\n\n\n> SELECT conv('100', 2, 10);\n 4\n> SELECT conv(-10, 16, -10);\n -16\n\n\n\n\n\n\ncorr\n\n\ncorr(expr1, expr2) - Returns Pearson coefficient of correlation between a set of number pairs.\n\n\n\n\ncos\n\n\ncos(expr) - Returns the cosine of \nexpr\n, as if computed by\n\njava.lang.Math.cos\n.\n\n\nArguments:\n\n\n\n\nexpr - angle in radians\n\n\n\n\nExamples:\n\n\n> SELECT cos(0);\n 1.0\n\n\n\n\n\n\ncosh\n\n\ncosh(expr) - Returns the hyperbolic cosine of \nexpr\n, as if computed by\n\njava.lang.Math.cosh\n.\n\n\nArguments:\n\n\n\n\nexpr - hyperbolic angle\n\n\n\n\nExamples:\n\n\n> SELECT cosh(0);\n 1.0\n\n\n\n\n\n\ncot\n\n\ncot(expr) - Returns the cotangent of \nexpr\n, as if computed by \n1/java.lang.Math.cot\n.\n\n\nArguments:\n\n\n\n\nexpr - angle in radians\n\n\n\n\nExamples:\n\n\n> SELECT cot(1);\n 0.6420926159343306\n\n\n\n\n\n\ncount\n\n\ncount(*) - Returns the total number of retrieved rows, including rows containing null.\n\n\ncount(expr) - Returns the number of rows for which the supplied expression is non-null.\n\n\ncount(DISTINCT expr[, expr...]) - Returns the number of rows for which the supplied expression(s) are unique and non-null.\n\n\n\n\ncount_min_sketch\n\n\ncount_min_sketch(col, eps, confidence, seed) - Returns a count-min sketch of a column with the given esp,\nconfidence and seed. The result is an array of bytes, which can be deserialized to a\n\nCountMinSketch\n before usage. Count-min sketch is a probabilistic data structure used for\ncardinality estimation using sub-linear space.\n\n\n\n\ncovar_pop\n\n\ncovar_pop(expr1, expr2) - Returns the population covariance of a set of number pairs.\n\n\n\n\ncovar_samp\n\n\ncovar_samp(expr1, expr2) - Returns the sample covariance of a set of number pairs.\n\n\n\n\ncrc32\n\n\ncrc32(expr) - Returns a cyclic redundancy check value of the \nexpr\n as a bigint.\n\n\nExamples:\n\n\n> SELECT crc32('Spark');\n 1557323817\n\n\n\n\n\n\ncube\n\n\ncube([col1[, col2 ..]]) - create a multi-dimensional cube using the specified columns\nso that we can run aggregation on them.\n\n\nExamples:\n\n\n> SELECT name, age, count(*) FROM VALUES (2, 'Alice'), (5, 'Bob') people(age, name) GROUP BY cube(name, age);\n NULL 2 1\n NULL NULL 2\n Alice 2 1\n Bob 5 1\n NULL 5 1\n Bob NULL 1\n Alice NULL 1\n\n\n\n\nSince:\n 2.0.0\n\n\n\n\ncume_dist\n\n\ncume_dist() - Computes the position of a value relative to all values in the partition.\n\n\n\n\ncurrent_database\n\n\ncurrent_database() - Returns the current database.\n\n\nExamples:\n\n\n> SELECT current_database();\n default\n\n\n\n\n\n\ncurrent_date\n\n\ncurrent_date() - Returns the current date at the start of query evaluation.\n\n\nSince:\n 1.5.0\n\n\n\n\ncurrent_timestamp\n\n\ncurrent_timestamp() - Returns the current timestamp at the start of query evaluation.\n\n\nSince:\n 1.5.0\n\n\n\n\ndate\n\n\ndate(expr) - Casts the value \nexpr\n to the target data type \ndate\n.\n\n\n\n\ndate_add\n\n\ndate_add(start_date, num_days) - Returns the date that is \nnum_days\n after \nstart_date\n.\n\n\nExamples:\n\n\n> SELECT date_add('2016-07-30', 1);\n 2016-07-31\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\ndate_format\n\n\ndate_format(timestamp, fmt) - Converts \ntimestamp\n to a value of string in the format specified by the date format \nfmt\n.\n\n\nExamples:\n\n\n> SELECT date_format('2016-04-08', 'y');\n 2016\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\ndate_sub\n\n\ndate_sub(start_date, num_days) - Returns the date that is \nnum_days\n before \nstart_date\n.\n\n\nExamples:\n\n\n> SELECT date_sub('2016-07-30', 1);\n 2016-07-29\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\ndate_trunc\n\n\ndate_trunc(fmt, ts) - Returns timestamp \nts\n truncated to the unit specified by the format model \nfmt\n.\n\nfmt\n should be one of [\"YEAR\", \"YYYY\", \"YY\", \"MON\", \"MONTH\", \"MM\", \"DAY\", \"DD\", \"HOUR\", \"MINUTE\", \"SECOND\", \"WEEK\", \"QUARTER\"]\n\n\nExamples:\n\n\n> SELECT date_trunc('YEAR', '2015-03-05T09:32:05.359');\n 2015-01-01 00:00:00\n> SELECT date_trunc('MM', '2015-03-05T09:32:05.359');\n 2015-03-01 00:00:00\n> SELECT date_trunc('DD', '2015-03-05T09:32:05.359');\n 2015-03-05 00:00:00\n> SELECT date_trunc('HOUR', '2015-03-05T09:32:05.359');\n 2015-03-05 09:00:00\n\n\n\n\nSince:\n 2.3.0\n\n\n\n\ndatediff\n\n\ndatediff(endDate, startDate) - Returns the number of days from \nstartDate\n to \nendDate\n.\n\n\nExamples:\n\n\n> SELECT datediff('2009-07-31', '2009-07-30');\n 1\n\n> SELECT datediff('2009-07-30', '2009-07-31');\n -1\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nday\n\n\nday(date) - Returns the day of month of the date/timestamp.\n\n\nExamples:\n\n\n> SELECT day('2009-07-30');\n 30\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\ndayofmonth\n\n\ndayofmonth(date) - Returns the day of month of the date/timestamp.\n\n\nExamples:\n\n\n> SELECT dayofmonth('2009-07-30');\n 30\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\ndayofweek\n\n\ndayofweek(date) - Returns the day of the week for date/timestamp (1 = Sunday, 2 = Monday, ..., 7 = Saturday).\n\n\nExamples:\n\n\n> SELECT dayofweek('2009-07-30');\n 5\n\n\n\n\nSince:\n 2.3.0\n\n\n\n\ndayofyear\n\n\ndayofyear(date) - Returns the day of year of the date/timestamp.\n\n\nExamples:\n\n\n> SELECT dayofyear('2016-04-09');\n 100\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\ndecimal\n\n\ndecimal(expr) - Casts the value \nexpr\n to the target data type \ndecimal\n.\n\n\n\n\ndecode\n\n\ndecode(bin, charset) - Decodes the first argument using the second argument character set.\n\n\nExamples:\n\n\n> SELECT decode(encode('abc', 'utf-8'), 'utf-8');\n abc\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\ndegrees\n\n\ndegrees(expr) - Converts radians to degrees.\n\n\nArguments:\n\n\n\n\nexpr - angle in radians\n\n\n\n\nExamples:\n\n\n> SELECT degrees(3.141592653589793);\n 180.0\n\n\n\n\n\n\ndense_rank\n\n\ndense_rank() - Computes the rank of a value in a group of values. The result is one plus the\npreviously assigned rank value. Unlike the function rank, dense_rank will not produce gaps\nin the ranking sequence.\n\n\n\n\ndouble\n\n\ndouble(expr) - Casts the value \nexpr\n to the target data type \ndouble\n.\n\n\n\n\ne\n\n\ne() - Returns Euler's number, e.\n\n\nExamples:\n\n\n> SELECT e();\n 2.718281828459045\n\n\n\n\n\n\nelt\n\n\nelt(n, input1, input2, ...) - Returns the \nn\n-th input, e.g., returns \ninput2\n when \nn\n is 2.\n\n\nExamples:\n\n\n> SELECT elt(1, 'scala', 'java');\n scala\n\n\n\n\nSince:\n 2.0.0\n\n\n\n\nencode\n\n\nencode(str, charset) - Encodes the first argument using the second argument character set.\n\n\nExamples:\n\n\n> SELECT encode('abc', 'utf-8');\n abc\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nexp\n\n\nexp(expr) - Returns e to the power of \nexpr\n.\n\n\nExamples:\n\n\n> SELECT exp(0);\n 1.0\n\n\n\n\n\n\nexplode\n\n\nexplode(expr) - Separates the elements of array \nexpr\n into multiple rows, or the elements of map \nexpr\n into multiple rows and columns.\n\n\nExamples:\n\n\n> SELECT explode(array(10, 20));\n 10\n 20\n\n\n\n\n\n\nexplode_outer\n\n\nexplode_outer(expr) - Separates the elements of array \nexpr\n into multiple rows, or the elements of map \nexpr\n into multiple rows and columns.\n\n\nExamples:\n\n\n> SELECT explode_outer(array(10, 20));\n 10\n 20\n\n\n\n\n\n\nexpm1\n\n\nexpm1(expr) - Returns exp(\nexpr\n) - 1.\n\n\nExamples:\n\n\n> SELECT expm1(0);\n 0.0\n\n\n\n\n\n\nfactorial\n\n\nfactorial(expr) - Returns the factorial of \nexpr\n. \nexpr\n is [0..20]. Otherwise, null.\n\n\nExamples:\n\n\n> SELECT factorial(5);\n 120\n\n\n\n\n\n\nfind_in_set\n\n\nfind_in_set(str, str_array) - Returns the index (1-based) of the given string (\nstr\n) in the comma-delimited list (\nstr_array\n).\nReturns 0, if the string was not found or if the given string (\nstr\n) contains a comma.\n\n\nExamples:\n\n\n> SELECT find_in_set('ab','abc,b,ab,c,def');\n 3\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nfirst\n\n\nfirst(expr[, isIgnoreNull]) - Returns the first value of \nexpr\n for a group of rows.\nIf \nisIgnoreNull\n is true, returns only non-null values.\n\n\n\n\nfirst_value\n\n\nfirst_value(expr[, isIgnoreNull]) - Returns the first value of \nexpr\n for a group of rows.\nIf \nisIgnoreNull\n is true, returns only non-null values.\n\n\n\n\nfloat\n\n\nfloat(expr) - Casts the value \nexpr\n to the target data type \nfloat\n.\n\n\n\n\nfloor\n\n\nfloor(expr) - Returns the largest integer not greater than \nexpr\n.\n\n\nExamples:\n\n\n> SELECT floor(-0.1);\n -1\n> SELECT floor(5);\n 5\n\n\n\n\n\n\nformat_number\n\n\nformat_number(expr1, expr2) - Formats the number \nexpr1\n like '#,###,###.##', rounded to \nexpr2\n\ndecimal places. If \nexpr2\n is 0, the result has no decimal point or fractional part.\nThis is supposed to function like MySQL's FORMAT.\n\n\nExamples:\n\n\n> SELECT format_number(12332.123456, 4);\n 12,332.1235\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nformat_string\n\n\nformat_string(strfmt, obj, ...) - Returns a formatted string from printf-style format strings.\n\n\nExamples:\n\n\n> SELECT format_string(\"Hello World %d %s\", 100, \"days\");\n Hello World 100 days\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nfrom_json\n\n\nfrom_json(jsonStr, schema[, options]) - Returns a struct value with the given \njsonStr\n and \nschema\n.\n\n\nExamples:\n\n\n> SELECT from_json('{\"a\":1, \"b\":0.8}', 'a INT, b DOUBLE');\n {\"a\":1, \"b\":0.8}\n> SELECT from_json('{\"time\":\"26/08/2015\"}', 'time Timestamp', map('timestampFormat', 'dd/MM/yyyy'));\n {\"time\":\"2015-08-26 00:00:00.0\"}\n\n\n\n\nSince:\n 2.2.0\n\n\n\n\nfrom_unixtime\n\n\nfrom_unixtime(unix_time, format) - Returns \nunix_time\n in the specified \nformat\n.\n\n\nExamples:\n\n\n> SELECT from_unixtime(0, 'yyyy-MM-dd HH:mm:ss');\n 1970-01-01 00:00:00\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nfrom_utc_timestamp\n\n\nfrom_utc_timestamp(timestamp, timezone) - Given a timestamp like '2017-07-14 02:40:00.0', interprets it as a time in UTC, and renders that time as a timestamp in the given time zone. For example, 'GMT+1' would yield '2017-07-14 03:40:00.0'.\n\n\nExamples:\n\n\n> SELECT from_utc_timestamp('2016-08-31', 'Asia/Seoul');\n 2016-08-31 09:00:00\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nget_json_object\n\n\nget_json_object(json_txt, path) - Extracts a json object from \npath\n.\n\n\nExamples:\n\n\n> SELECT get_json_object('{\"a\":\"b\"}', '$.a');\n b\n\n\n\n\n\n\ngreatest\n\n\ngreatest(expr, ...) - Returns the greatest value of all parameters, skipping null values.\n\n\nExamples:\n\n\n> SELECT greatest(10, 9, 2, 4, 3);\n 10\n\n\n\n\n\n\ngrouping\n\n\ngrouping(col) - indicates whether a specified column in a GROUP BY is aggregated or\nnot, returns 1 for aggregated or 0 for not aggregated in the result set.\",\n\n\nExamples:\n\n\n> SELECT name, grouping(name), sum(age) FROM VALUES (2, 'Alice'), (5, 'Bob') people(age, name) GROUP BY cube(name);\n Alice 0 2\n NULL 1 7\n Bob 0 5\n\n\n\n\nSince:\n 2.0.0\n\n\n\n\ngrouping_id\n\n\ngrouping_id([col1[, col2 ..]]) - returns the level of grouping, equals to\n\n(grouping(c1) << (n-1)) + (grouping(c2) << (n-2)) + ... + grouping(cn)\n\n\nExamples:\n\n\n> SELECT name, grouping_id(), sum(age), avg(height) FROM VALUES (2, 'Alice', 165), (5, 'Bob', 180) people(age, name, height) GROUP BY cube(name, height);\n NULL 2 2 165.0\n Alice 0 2 165.0\n NULL 2 5 180.0\n NULL 3 7 172.5\n Bob 0 5 180.0\n Bob 1 5 180.0\n Alice 1 2 165.0\n\n\n\n\nNote:\n\n\nInput columns should match with grouping columns exactly, or empty (means all the grouping\ncolumns).\n\n\nSince:\n 2.0.0\n\n\n\n\nhash\n\n\nhash(expr1, expr2, ...) - Returns a hash value of the arguments.\n\n\nExamples:\n\n\n> SELECT hash('Spark', array(123), 2);\n -1321691492\n\n\n\n\n\n\nhex\n\n\nhex(expr) - Converts \nexpr\n to hexadecimal.\n\n\nExamples:\n\n\n> SELECT hex(17);\n 11\n> SELECT hex('Spark SQL');\n 537061726B2053514C\n\n\n\n\n\n\nhour\n\n\nhour(timestamp) - Returns the hour component of the string/timestamp.\n\n\nExamples:\n\n\n> SELECT hour('2009-07-30 12:58:59');\n 12\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nhypot\n\n\nhypot(expr1, expr2) - Returns sqrt(\nexpr1\n2 + \nexpr2\n2).\n\n\nExamples:\n\n\n> SELECT hypot(3, 4);\n 5.0\n\n\n\n\n\n\nif\n\n\nif(expr1, expr2, expr3) - If \nexpr1\n evaluates to true, then returns \nexpr2\n; otherwise returns \nexpr3\n.\n\n\nExamples:\n\n\n> SELECT if(1 < 2, 'a', 'b');\n a\n\n\n\n\n\n\nifnull\n\n\nifnull(expr1, expr2) - Returns \nexpr2\n if \nexpr1\n is null, or \nexpr1\n otherwise.\n\n\nExamples:\n\n\n> SELECT ifnull(NULL, array('2'));\n [\"2\"]\n\n\n\n\nSince:\n 2.0.0\n\n\n\n\nin\n\n\nexpr1 in(expr2, expr3, ...) - Returns true if \nexpr\n equals to any valN.\n\n\nArguments:\n\n\n\n\nexpr1, expr2, expr3, ... - the arguments must be same type.\n\n\n\n\nExamples:\n\n\n> SELECT 1 in(1, 2, 3);\n true\n> SELECT 1 in(2, 3, 4);\n false\n> SELECT named_struct('a', 1, 'b', 2) in(named_struct('a', 1, 'b', 1), named_struct('a', 1, 'b', 3));\n false\n> SELECT named_struct('a', 1, 'b', 2) in(named_struct('a', 1, 'b', 2), named_struct('a', 1, 'b', 3));\n true\n\n\n\n\n\n\ninitcap\n\n\ninitcap(str) - Returns \nstr\n with the first letter of each word in uppercase.\nAll other letters are in lowercase. Words are delimited by white space.\n\n\nExamples:\n\n\n> SELECT initcap('sPark sql');\n Spark Sql\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\ninline\n\n\ninline(expr) - Explodes an array of structs into a table.\n\n\nExamples:\n\n\n> SELECT inline(array(struct(1, 'a'), struct(2, 'b')));\n 1 a\n 2 b\n\n\n\n\n\n\ninline_outer\n\n\ninline_outer(expr) - Explodes an array of structs into a table.\n\n\nExamples:\n\n\n> SELECT inline_outer(array(struct(1, 'a'), struct(2, 'b')));\n 1 a\n 2 b\n\n\n\n\n\n\ninput_file_block_length\n\n\ninput_file_block_length() - Returns the length of the block being read, or -1 if not available.\n\n\n\n\ninput_file_block_start\n\n\ninput_file_block_start() - Returns the start offset of the block being read, or -1 if not available.\n\n\n\n\ninput_file_name\n\n\ninput_file_name() - Returns the name of the file being read, or empty string if not available.\n\n\n\n\ninstr\n\n\ninstr(str, substr) - Returns the (1-based) index of the first occurrence of \nsubstr\n in \nstr\n.\n\n\nExamples:\n\n\n> SELECT instr('SparkSQL', 'SQL');\n 6\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nint\n\n\nint(expr) - Casts the value \nexpr\n to the target data type \nint\n.\n\n\n\n\nisnan\n\n\nisnan(expr) - Returns true if \nexpr\n is NaN, or false otherwise.\n\n\nExamples:\n\n\n> SELECT isnan(cast('NaN' as double));\n true\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nisnotnull\n\n\nisnotnull(expr) - Returns true if \nexpr\n is not null, or false otherwise.\n\n\nExamples:\n\n\n> SELECT isnotnull(1);\n true\n\n\n\n\nSince:\n 1.0.0\n\n\n\n\nisnull\n\n\nisnull(expr) - Returns true if \nexpr\n is null, or false otherwise.\n\n\nExamples:\n\n\n> SELECT isnull(1);\n false\n\n\n\n\nSince:\n 1.0.0\n\n\n\n\njava_method\n\n\njava_method(class, method[, arg1[, arg2 ..]]) - Calls a method with reflection.\n\n\nExamples:\n\n\n> SELECT java_method('java.util.UUID', 'randomUUID');\n c33fb387-8500-4bfa-81d2-6e0e3e930df2\n> SELECT java_method('java.util.UUID', 'fromString', 'a5cf6c42-0c85-418f-af6c-3e4e5b1328f2');\n a5cf6c42-0c85-418f-af6c-3e4e5b1328f2\n\n\n\n\n\n\njson_tuple\n\n\njson_tuple(jsonStr, p1, p2, ..., pn) - Returns a tuple like the function get_json_object, but it takes multiple names. All the input parameters and output column types are string.\n\n\nExamples:\n\n\n> SELECT json_tuple('{\"a\":1, \"b\":2}', 'a', 'b');\n 1 2\n\n\n\n\n\n\nkurtosis\n\n\nkurtosis(expr) - Returns the kurtosis value calculated from values of a group.\n\n\n\n\nlag\n\n\nlag(input[, offset[, default]]) - Returns the value of \ninput\n at the \noffset\nth row\nbefore the current row in the window. The default value of \noffset\n is 1 and the default\nvalue of \ndefault\n is null. If the value of \ninput\n at the \noffset\nth row is null,\nnull is returned. If there is no such offset row (e.g., when the offset is 1, the first\nrow of the window does not have any previous row), \ndefault\n is returned.\n\n\n\n\nlast\n\n\nlast(expr[, isIgnoreNull]) - Returns the last value of \nexpr\n for a group of rows.\nIf \nisIgnoreNull\n is true, returns only non-null values.\n\n\n\n\nlast_day\n\n\nlast_day(date) - Returns the last day of the month which the date belongs to.\n\n\nExamples:\n\n\n> SELECT last_day('2009-01-12');\n 2009-01-31\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nlast_value\n\n\nlast_value(expr[, isIgnoreNull]) - Returns the last value of \nexpr\n for a group of rows.\nIf \nisIgnoreNull\n is true, returns only non-null values.\n\n\n\n\nlcase\n\n\nlcase(str) - Returns \nstr\n with all characters changed to lowercase.\n\n\nExamples:\n\n\n> SELECT lcase('SparkSql');\n sparksql\n\n\n\n\nSince:\n 1.0.1\n\n\n\n\nlead\n\n\nlead(input[, offset[, default]]) - Returns the value of \ninput\n at the \noffset\nth row\nafter the current row in the window. The default value of \noffset\n is 1 and the default\nvalue of \ndefault\n is null. If the value of \ninput\n at the \noffset\nth row is null,\nnull is returned. If there is no such an offset row (e.g., when the offset is 1, the last\nrow of the window does not have any subsequent row), \ndefault\n is returned.\n\n\n\n\nleast\n\n\nleast(expr, ...) - Returns the least value of all parameters, skipping null values.\n\n\nExamples:\n\n\n> SELECT least(10, 9, 2, 4, 3);\n 2\n\n\n\n\n\n\nleft\n\n\nleft(str, len) - Returns the leftmost \nlen\n(\nlen\n can be string type) characters from the string \nstr\n,if \nlen\n is less or equal than 0 the result is an empty string.\n\n\nExamples:\n\n\n> SELECT left('Spark SQL', 3);\n Spa\n\n\n\n\nSince:\n 2.3.0\n\n\n\n\nlength\n\n\nlength(expr) - Returns the character length of string data or number of bytes of binary data. The length of string data includes the trailing spaces. The length of binary data includes binary zeros.\n\n\nExamples:\n\n\n> SELECT length('Spark SQL ');\n 10\n> SELECT CHAR_LENGTH('Spark SQL ');\n 10\n> SELECT CHARACTER_LENGTH('Spark SQL ');\n 10\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nlevenshtein\n\n\nlevenshtein(str1, str2) - Returns the Levenshtein distance between the two given strings.\n\n\nExamples:\n\n\n> SELECT levenshtein('kitten', 'sitting');\n 3\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nlike\n\n\nstr like pattern - Returns true if str matches pattern, null if any arguments are null, false otherwise.\n\n\nArguments:\n\n\n\n\nstr - a string expression\n\n\n\n\npattern - a string expression. The pattern is a string which is matched literally, with\n exception to the following special symbols:\n\n\n_ matches any one character in the input (similar to . in posix regular expressions)\n\n\n% matches zero or more characters in the input (similar to .* in posix regular\nexpressions)\n\n\nThe escape character is '\\'. If an escape character precedes a special symbol or another\nescape character, the following character is matched literally. It is invalid to escape\nany other character.\n\n\nSince Spark 2.0, string literals are unescaped in our SQL parser. For example, in order\nto match \"\\abc\", the pattern should be \"\\abc\".\n\n\nWhen SQL config 'spark.sql.parser.escapedStringLiterals' is enabled, it fallbacks\nto Spark 1.6 behavior regarding string literal parsing. For example, if the config is\nenabled, the pattern to match \"\\abc\" should be \"\\abc\".\n\n\n\n\n\n\nExamples:\n\n\n> SELECT '%SystemDrive%\\Users\\John' like '\\%SystemDrive\\%\\\\Users%'\ntrue\n\n\n\n\nNote:\n\n\nUse RLIKE to match with standard regular expressions.\n\n\nSince:\n 1.0.0\n\n\n\n\nln\n\n\nln(expr) - Returns the natural logarithm (base e) of \nexpr\n.\n\n\nExamples:\n\n\n> SELECT ln(1);\n 0.0\n\n\n\n\n\n\nlocate\n\n\nlocate(substr, str[, pos]) - Returns the position of the first occurrence of \nsubstr\n in \nstr\n after position \npos\n.\nThe given \npos\n and return value are 1-based.\n\n\nExamples:\n\n\n> SELECT locate('bar', 'foobarbar');\n 4\n> SELECT locate('bar', 'foobarbar', 5);\n 7\n> SELECT POSITION('bar' IN 'foobarbar');\n 4\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nlog\n\n\nlog(base, expr) - Returns the logarithm of \nexpr\n with \nbase\n.\n\n\nExamples:\n\n\n> SELECT log(10, 100);\n 2.0\n\n\n\n\n\n\nlog10\n\n\nlog10(expr) - Returns the logarithm of \nexpr\n with base 10.\n\n\nExamples:\n\n\n> SELECT log10(10);\n 1.0\n\n\n\n\n\n\nlog1p\n\n\nlog1p(expr) - Returns log(1 + \nexpr\n).\n\n\nExamples:\n\n\n> SELECT log1p(0);\n 0.0\n\n\n\n\n\n\nlog2\n\n\nlog2(expr) - Returns the logarithm of \nexpr\n with base 2.\n\n\nExamples:\n\n\n> SELECT log2(2);\n 1.0\n\n\n\n\n\n\nlower\n\n\nlower(str) - Returns \nstr\n with all characters changed to lowercase.\n\n\nExamples:\n\n\n> SELECT lower('SparkSql');\n sparksql\n\n\n\n\nSince:\n 1.0.1\n\n\n\n\nlpad\n\n\nlpad(str, len, pad) - Returns \nstr\n, left-padded with \npad\n to a length of \nlen\n.\nIf \nstr\n is longer than \nlen\n, the return value is shortened to \nlen\n characters.\n\n\nExamples:\n\n\n> SELECT lpad('hi', 5, '??');\n ???hi\n> SELECT lpad('hi', 1, '??');\n h\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nltrim\n\n\nltrim(str) - Removes the leading space characters from \nstr\n.\n\n\nltrim(trimStr, str) - Removes the leading string contains the characters from the trim string\n\n\nArguments:\n\n\n\n\nstr - a string expression\n\n\ntrimStr - the trim string characters to trim, the default value is a single space\n\n\n\n\nExamples:\n\n\n> SELECT ltrim(' SparkSQL ');\n SparkSQL\n> SELECT ltrim('Sp', 'SSparkSQLS');\n arkSQLS\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nmap\n\n\nmap(key0, value0, key1, value1, ...) - Creates a map with the given key/value pairs.\n\n\nExamples:\n\n\n> SELECT map(1.0, '2', 3.0, '4');\n {1.0:\"2\",3.0:\"4\"}\n\n\n\n\n\n\nmap_keys\n\n\nmap_keys(map) - Returns an unordered array containing the keys of the map.\n\n\nExamples:\n\n\n> SELECT map_keys(map(1, 'a', 2, 'b'));\n [1,2]\n\n\n\n\n\n\nmap_values\n\n\nmap_values(map) - Returns an unordered array containing the values of the map.\n\n\nExamples:\n\n\n> SELECT map_values(map(1, 'a', 2, 'b'));\n [\"a\",\"b\"]\n\n\n\n\n\n\nmax\n\n\nmax(expr) - Returns the maximum value of \nexpr\n.\n\n\n\n\nmd5\n\n\nmd5(expr) - Returns an MD5 128-bit checksum as a hex string of \nexpr\n.\n\n\nExamples:\n\n\n> SELECT md5('Spark');\n 8cde774d6f7333752ed72cacddb05126\n\n\n\n\n\n\nmean\n\n\nmean(expr) - Returns the mean calculated from values of a group.\n\n\n\n\nmin\n\n\nmin(expr) - Returns the minimum value of \nexpr\n.\n\n\n\n\nminute\n\n\nminute(timestamp) - Returns the minute component of the string/timestamp.\n\n\nExamples:\n\n\n> SELECT minute('2009-07-30 12:58:59');\n 58\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nmod\n\n\nexpr1 mod expr2 - Returns the remainder after \nexpr1\n/\nexpr2\n.\n\n\nExamples:\n\n\n> SELECT 2 mod 1.8;\n 0.2\n> SELECT MOD(2, 1.8);\n 0.2\n\n\n\n\n\n\nmonotonically_increasing_id\n\n\nmonotonically_increasing_id() - Returns monotonically increasing 64-bit integers. The generated ID is guaranteed\nto be monotonically increasing and unique, but not consecutive. The current implementation\nputs the partition ID in the upper 31 bits, and the lower 33 bits represent the record number\nwithin each partition. The assumption is that the data frame has less than 1 billion\npartitions, and each partition has less than 8 billion records.\n\n\n\n\nmonth\n\n\nmonth(date) - Returns the month component of the date/timestamp.\n\n\nExamples:\n\n\n> SELECT month('2016-07-30');\n 7\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nmonths_between\n\n\nmonths_between(timestamp1, timestamp2) - Returns number of months between \ntimestamp1\n and \ntimestamp2\n.\n\n\nExamples:\n\n\n> SELECT months_between('1997-02-28 10:30:00', '1996-10-30');\n 3.94959677\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nnamed_struct\n\n\nnamed_struct(name1, val1, name2, val2, ...) - Creates a struct with the given field names and values.\n\n\nExamples:\n\n\n> SELECT named_struct(\"a\", 1, \"b\", 2, \"c\", 3);\n {\"a\":1,\"b\":2,\"c\":3}\n\n\n\n\n\n\nnanvl\n\n\nnanvl(expr1, expr2) - Returns \nexpr1\n if it's not NaN, or \nexpr2\n otherwise.\n\n\nExamples:\n\n\n> SELECT nanvl(cast('NaN' as double), 123);\n 123.0\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nnegative\n\n\nnegative(expr) - Returns the negated value of \nexpr\n.\n\n\nExamples:\n\n\n> SELECT negative(1);\n -1\n\n\n\n\n\n\nnext_day\n\n\nnext_day(start_date, day_of_week) - Returns the first date which is later than \nstart_date\n and named as indicated.\n\n\nExamples:\n\n\n> SELECT next_day('2015-01-14', 'TU');\n 2015-01-20\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nnot\n\n\nnot expr - Logical not.\n\n\n\n\nnow\n\n\nnow() - Returns the current timestamp at the start of query evaluation.\n\n\nSince:\n 1.5.0\n\n\n\n\nntile\n\n\nntile(n) - Divides the rows for each window partition into \nn\n buckets ranging\nfrom 1 to at most \nn\n.\n\n\n\n\nnullif\n\n\nnullif(expr1, expr2) - Returns null if \nexpr1\n equals to \nexpr2\n, or \nexpr1\n otherwise.\n\n\nExamples:\n\n\n> SELECT nullif(2, 2);\n NULL\n\n\n\n\nSince:\n 2.0.0\n\n\n\n\nnvl\n\n\nnvl(expr1, expr2) - Returns \nexpr2\n if \nexpr1\n is null, or \nexpr1\n otherwise.\n\n\nExamples:\n\n\n> SELECT nvl(NULL, array('2'));\n [\"2\"]\n\n\n\n\nSince:\n 2.0.0\n\n\n\n\nnvl2\n\n\nnvl2(expr1, expr2, expr3) - Returns \nexpr2\n if \nexpr1\n is not null, or \nexpr3\n otherwise.\n\n\nExamples:\n\n\n> SELECT nvl2(NULL, 2, 1);\n 1\n\n\n\n\nSince:\n 2.0.0\n\n\n\n\noctet_length\n\n\noctet_length(expr) - Returns the byte length of string data or number of bytes of binary data.\n\n\nExamples:\n\n\n> SELECT octet_length('Spark SQL');\n 9\n\n\n\n\nSince:\n 2.3.0\n\n\n\n\nor\n\n\nexpr1 or expr2 - Logical OR.\n\n\n\n\nparse_url\n\n\nparse_url(url, partToExtract[, key]) - Extracts a part from a URL.\n\n\nExamples:\n\n\n> SELECT parse_url('
http://spark.apache.org/path?query=1', 'HOST')\n spark.apache.org\n> SELECT parse_url('
http://spark.apache.org/path?query=1', 'QUERY')\n query=1\n> SELECT parse_url('
http://spark.apache.org/path?query=1', 'QUERY', 'query')\n 1\n\n\n\n\nSince:\n 2.0.0\n\n\n\n\npercent_rank\n\n\npercent_rank() - Computes the percentage ranking of a value in a group of values.\n\n\n\n\npercentile\n\n\npercentile(col, percentage [, frequency]) - Returns the exact percentile value of numeric column\n\ncol\n at the given percentage. The value of percentage must be between 0.0 and 1.0. The\nvalue of frequency should be positive integral\n\n\npercentile(col, array(percentage1 [, percentage2]...) [, frequency]) - Returns the exact\npercentile value array of numeric column \ncol\n at the given percentage(s). Each value\nof the percentage array must be between 0.0 and 1.0. The value of frequency should be\npositive integral\n\n\n\n\npercentile_approx\n\n\npercentile_approx(col, percentage [, accuracy]) - Returns the approximate percentile value of numeric\ncolumn \ncol\n at the given percentage. The value of percentage must be between 0.0\nand 1.0. The \naccuracy\n parameter (default: 10000) is a positive numeric literal which\ncontrols approximation accuracy at the cost of memory. Higher value of \naccuracy\n yields\nbetter accuracy, \n1.0/accuracy\n is the relative error of the approximation.\nWhen \npercentage\n is an array, each value of the percentage array must be between 0.0 and 1.0.\nIn this case, returns the approximate percentile array of column \ncol\n at the given\npercentage array.\n\n\nExamples:\n\n\n> SELECT percentile_approx(10.0, array(0.5, 0.4, 0.1), 100);\n [10.0,10.0,10.0]\n> SELECT percentile_approx(10.0, 0.5, 100);\n 10.0\n\n\n\n\n\n\npi\n\n\npi() - Returns pi.\n\n\nExamples:\n\n\n> SELECT pi();\n 3.141592653589793\n\n\n\n\n\n\npmod\n\n\npmod(expr1, expr2) - Returns the positive value of \nexpr1\n mod \nexpr2\n.\n\n\nExamples:\n\n\n> SELECT pmod(10, 3);\n 1\n> SELECT pmod(-10, 3);\n 2\n\n\n\n\n\n\nposexplode\n\n\nposexplode(expr) - Separates the elements of array \nexpr\n into multiple rows with positions, or the elements of map \nexpr\n into multiple rows and columns with positions.\n\n\nExamples:\n\n\n> SELECT posexplode(array(10,20));\n 0 10\n 1 20\n\n\n\n\n\n\nposexplode_outer\n\n\nposexplode_outer(expr) - Separates the elements of array \nexpr\n into multiple rows with positions, or the elements of map \nexpr\n into multiple rows and columns with positions.\n\n\nExamples:\n\n\n> SELECT posexplode_outer(array(10,20));\n 0 10\n 1 20\n\n\n\n\n\n\nposition\n\n\nposition(substr, str[, pos]) - Returns the position of the first occurrence of \nsubstr\n in \nstr\n after position \npos\n.\nThe given \npos\n and return value are 1-based.\n\n\nExamples:\n\n\n> SELECT position('bar', 'foobarbar');\n 4\n> SELECT position('bar', 'foobarbar', 5);\n 7\n> SELECT POSITION('bar' IN 'foobarbar');\n 4\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\npositive\n\n\npositive(expr) - Returns the value of \nexpr\n.\n\n\n\n\npow\n\n\npow(expr1, expr2) - Raises \nexpr1\n to the power of \nexpr2\n.\n\n\nExamples:\n\n\n> SELECT pow(2, 3);\n 8.0\n\n\n\n\n\n\npower\n\n\npower(expr1, expr2) - Raises \nexpr1\n to the power of \nexpr2\n.\n\n\nExamples:\n\n\n> SELECT power(2, 3);\n 8.0\n\n\n\n\n\n\nprintf\n\n\nprintf(strfmt, obj, ...) - Returns a formatted string from printf-style format strings.\n\n\nExamples:\n\n\n> SELECT printf(\"Hello World %d %s\", 100, \"days\");\n Hello World 100 days\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nquarter\n\n\nquarter(date) - Returns the quarter of the year for date, in the range 1 to 4.\n\n\nExamples:\n\n\n> SELECT quarter('2016-08-31');\n 3\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nradians\n\n\nradians(expr) - Converts degrees to radians.\n\n\nArguments:\n\n\n\n\nexpr - angle in degrees\n\n\n\n\nExamples:\n\n\n> SELECT radians(180);\n 3.141592653589793\n\n\n\n\n\n\nrand\n\n\nrand([seed]) - Returns a random value with independent and identically distributed (i.i.d.) uniformly distributed values in [0, 1).\n\n\nExamples:\n\n\n> SELECT rand();\n 0.9629742951434543\n> SELECT rand(0);\n 0.8446490682263027\n> SELECT rand(null);\n 0.8446490682263027\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nrandn\n\n\nrandn([seed]) - Returns a random value with independent and identically distributed (i.i.d.) values drawn from the standard normal distribution.\n\n\nExamples:\n\n\n> SELECT randn();\n -0.3254147983080288\n> SELECT randn(0);\n 1.1164209726833079\n> SELECT randn(null);\n 1.1164209726833079\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nrank\n\n\nrank() - Computes the rank of a value in a group of values. The result is one plus the number\nof rows preceding or equal to the current row in the ordering of the partition. The values\nwill produce gaps in the sequence.\n\n\n\n\nreflect\n\n\nreflect(class, method[, arg1[, arg2 ..]]) - Calls a method with reflection.\n\n\nExamples:\n\n\n> SELECT reflect('java.util.UUID', 'randomUUID');\n c33fb387-8500-4bfa-81d2-6e0e3e930df2\n> SELECT reflect('java.util.UUID', 'fromString', 'a5cf6c42-0c85-418f-af6c-3e4e5b1328f2');\n a5cf6c42-0c85-418f-af6c-3e4e5b1328f2\n\n\n\n\n\n\nregexp_extract\n\n\nregexp_extract(str, regexp[, idx]) - Extracts a group that matches \nregexp\n.\n\n\nExamples:\n\n\n> SELECT regexp_extract('100-200', '(\\d+)-(\\d+)', 1);\n 100\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nregexp_replace\n\n\nregexp_replace(str, regexp, rep) - Replaces all substrings of \nstr\n that match \nregexp\n with \nrep\n.\n\n\nExamples:\n\n\n> SELECT regexp_replace('100-200', '(\\d+)', 'num');\n num-num\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nrepeat\n\n\nrepeat(str, n) - Returns the string which repeats the given string value n times.\n\n\nExamples:\n\n\n> SELECT repeat('123', 2);\n 123123\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nreplace\n\n\nreplace(str, search[, replace]) - Replaces all occurrences of \nsearch\n with \nreplace\n.\n\n\nArguments:\n\n\n\n\nstr - a string expression\n\n\nsearch - a string expression. If \nsearch\n is not found in \nstr\n, \nstr\n is returned unchanged.\n\n\nreplace - a string expression. If \nreplace\n is not specified or is an empty string, nothing replaces\n the string that is removed from \nstr\n.\n\n\n\n\nExamples:\n\n\n> SELECT replace('ABCabc', 'abc', 'DEF');\n ABCDEF\n\n\n\n\nSince:\n 2.3.0\n\n\n\n\nreverse\n\n\nreverse(str) - Returns the reversed given string.\n\n\nExamples:\n\n\n> SELECT reverse('Spark SQL');\n LQS krapS\n\n\n\n\n\n\nright\n\n\nright(str, len) - Returns the rightmost \nlen\n(\nlen\n can be string type) characters from the string \nstr\n,if \nlen\n is less or equal than 0 the result is an empty string.\n\n\nExamples:\n\n\n> SELECT right('Spark SQL', 3);\n SQL\n\n\n\n\nSince:\n 2.3.0\n\n\n\n\nrint\n\n\nrint(expr) - Returns the double value that is closest in value to the argument and is equal to a mathematical integer.\n\n\nExamples:\n\n\n> SELECT rint(12.3456);\n 12.0\n\n\n\n\n\n\nrlike\n\n\nstr rlike regexp - Returns true if \nstr\n matches \nregexp\n, or false otherwise.\n\n\nArguments:\n\n\n\n\nstr - a string expression\n\n\n\n\nregexp - a string expression. The pattern string should be a Java regular expression.\n\n\nSince Spark 2.0, string literals (including regex patterns) are unescaped in our SQL\nparser. For example, to match \"\\abc\", a regular expression for \nregexp\n can be\n\"^\\abc$\".\n\n\nThere is a SQL config 'spark.sql.parser.escapedStringLiterals' that can be used to\nfallback to the Spark 1.6 behavior regarding string literal parsing. For example,\nif the config is enabled, the \nregexp\n that can match \"\\abc\" is \"^\\abc$\".\n\n\n\n\n\n\nExamples:\n\n\nWhen spark.sql.parser.escapedStringLiterals is disabled (default).\n> SELECT '%SystemDrive%\\Users\\John' rlike '%SystemDrive%\\\\Users.*'\ntrue\n\nWhen spark.sql.parser.escapedStringLiterals is enabled.\n> SELECT '%SystemDrive%\\Users\\John' rlike '%SystemDrive%\\Users.*'\ntrue\n\n\n\n\nNote:\n\n\nUse LIKE to match with simple string pattern.\n\n\nSince:\n 1.0.0\n\n\n\n\nrollup\n\n\nrollup([col1[, col2 ..]]) - create a multi-dimensional rollup using the specified columns\nso that we can run aggregation on them.\n\n\nExamples:\n\n\n> SELECT name, age, count(*) FROM VALUES (2, 'Alice'), (5, 'Bob') people(age, name) GROUP BY rollup(name, age);\n NULL NULL 2\n Alice 2 1\n Bob 5 1\n Bob NULL 1\n Alice NULL 1\n\n\n\n\nSince:\n 2.0.0\n\n\n\n\nround\n\n\nround(expr, d) - Returns \nexpr\n rounded to \nd\n decimal places using HALF_UP rounding mode.\n\n\nExamples:\n\n\n> SELECT round(2.5, 0);\n 3.0\n\n\n\n\n\n\nrow_number\n\n\nrow_number() - Assigns a unique, sequential number to each row, starting with one,\naccording to the ordering of rows within the window partition.\n\n\n\n\nrpad\n\n\nrpad(str, len, pad) - Returns \nstr\n, right-padded with \npad\n to a length of \nlen\n.\nIf \nstr\n is longer than \nlen\n, the return value is shortened to \nlen\n characters.\n\n\nExamples:\n\n\n> SELECT rpad('hi', 5, '??');\n hi???\n> SELECT rpad('hi', 1, '??');\n h\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nrtrim\n\n\nrtrim(str) - Removes the trailing space characters from \nstr\n.\n\n\nrtrim(trimStr, str) - Removes the trailing string which contains the characters from the trim string from the \nstr\n\n\nArguments:\n\n\n\n\nstr - a string expression\n\n\ntrimStr - the trim string characters to trim, the default value is a single space\n\n\n\n\nExamples:\n\n\n> SELECT rtrim(' SparkSQL ');\n SparkSQL\n> SELECT rtrim('LQSa', 'SSparkSQLS');\n SSpark\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nsecond\n\n\nsecond(timestamp) - Returns the second component of the string/timestamp.\n\n\nExamples:\n\n\n> SELECT second('2009-07-30 12:58:59');\n 59\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nsentences\n\n\nsentences(str[, lang, country]) - Splits \nstr\n into an array of array of words.\n\n\nExamples:\n\n\n> SELECT sentences('Hi there! Good morning.');\n [[\"Hi\",\"there\"],[\"Good\",\"morning\"]]\n\n\n\n\nSince:\n 2.0.0\n\n\n\n\nsha\n\n\nsha(expr) - Returns a sha1 hash value as a hex string of the \nexpr\n.\n\n\nExamples:\n\n\n> SELECT sha('Spark');\n 85f5955f4b27a9a4c2aab6ffe5d7189fc298b92c\n\n\n\n\n\n\nsha1\n\n\nsha1(expr) - Returns a sha1 hash value as a hex string of the \nexpr\n.\n\n\nExamples:\n\n\n> SELECT sha1('Spark');\n 85f5955f4b27a9a4c2aab6ffe5d7189fc298b92c\n\n\n\n\n\n\nsha2\n\n\nsha2(expr, bitLength) - Returns a checksum of SHA-2 family as a hex string of \nexpr\n.\nSHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256.\n\n\nExamples:\n\n\n> SELECT sha2('Spark', 256);\n 529bc3b07127ecb7e53a4dcf1991d9152c24537d919178022b2c42657f79a26b\n\n\n\n\n\n\nshiftleft\n\n\nshiftleft(base, expr) - Bitwise left shift.\n\n\nExamples:\n\n\n> SELECT shiftleft(2, 1);\n 4\n\n\n\n\n\n\nshiftright\n\n\nshiftright(base, expr) - Bitwise (signed) right shift.\n\n\nExamples:\n\n\n> SELECT shiftright(4, 1);\n 2\n\n\n\n\n\n\nshiftrightunsigned\n\n\nshiftrightunsigned(base, expr) - Bitwise unsigned right shift.\n\n\nExamples:\n\n\n> SELECT shiftrightunsigned(4, 1);\n 2\n\n\n\n\n\n\nsign\n\n\nsign(expr) - Returns -1.0, 0.0 or 1.0 as \nexpr\n is negative, 0 or positive.\n\n\nExamples:\n\n\n> SELECT sign(40);\n 1.0\n\n\n\n\n\n\nsignum\n\n\nsignum(expr) - Returns -1.0, 0.0 or 1.0 as \nexpr\n is negative, 0 or positive.\n\n\nExamples:\n\n\n> SELECT signum(40);\n 1.0\n\n\n\n\n\n\nsin\n\n\nsin(expr) - Returns the sine of \nexpr\n, as if computed by \njava.lang.Math.sin\n.\n\n\nArguments:\n\n\n\n\nexpr - angle in radians\n\n\n\n\nExamples:\n\n\n> SELECT sin(0);\n 0.0\n\n\n\n\n\n\nsinh\n\n\nsinh(expr) - Returns hyperbolic sine of \nexpr\n, as if computed by \njava.lang.Math.sinh\n.\n\n\nArguments:\n\n\n\n\nexpr - hyperbolic angle\n\n\n\n\nExamples:\n\n\n> SELECT sinh(0);\n 0.0\n\n\n\n\n\n\nsize\n\n\nsize(expr) - Returns the size of an array or a map. Returns -1 if null.\n\n\nExamples:\n\n\n> SELECT size(array('b', 'd', 'c', 'a'));\n 4\n\n\n\n\n\n\nskewness\n\n\nskewness(expr) - Returns the skewness value calculated from values of a group.\n\n\n\n\nsmallint\n\n\nsmallint(expr) - Casts the value \nexpr\n to the target data type \nsmallint\n.\n\n\n\n\nsort_array\n\n\nsort_array(array[, ascendingOrder]) - Sorts the input array in ascending or descending order according to the natural ordering of the array elements.\n\n\nExamples:\n\n\n> SELECT sort_array(array('b', 'd', 'c', 'a'), true);\n [\"a\",\"b\",\"c\",\"d\"]\n\n\n\n\n\n\nsoundex\n\n\nsoundex(str) - Returns Soundex code of the string.\n\n\nExamples:\n\n\n> SELECT soundex('Miller');\n M460\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nspace\n\n\nspace(n) - Returns a string consisting of \nn\n spaces.\n\n\nExamples:\n\n\n> SELECT concat(space(2), '1');\n 1\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nspark_partition_id\n\n\nspark_partition_id() - Returns the current partition id.\n\n\n\n\nsplit\n\n\nsplit(str, regex) - Splits \nstr\n around occurrences that match \nregex\n.\n\n\nExamples:\n\n\n> SELECT split('oneAtwoBthreeC', '[ABC]');\n [\"one\",\"two\",\"three\",\"\"]\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nsqrt\n\n\nsqrt(expr) - Returns the square root of \nexpr\n.\n\n\nExamples:\n\n\n> SELECT sqrt(4);\n 2.0\n\n\n\n\n\n\nstack\n\n\nstack(n, expr1, ..., exprk) - Separates \nexpr1\n, ..., \nexprk\n into \nn\n rows.\n\n\nExamples:\n\n\n> SELECT stack(2, 1, 2, 3);\n 1 2\n 3 NULL\n\n\n\n\n\n\nstd\n\n\nstd(expr) - Returns the sample standard deviation calculated from values of a group.\n\n\n\n\nstddev\n\n\nstddev(expr) - Returns the sample standard deviation calculated from values of a group.\n\n\n\n\nstddev_pop\n\n\nstddev_pop(expr) - Returns the population standard deviation calculated from values of a group.\n\n\n\n\nstddev_samp\n\n\nstddev_samp(expr) - Returns the sample standard deviation calculated from values of a group.\n\n\n\n\nstr_to_map\n\n\nstr_to_map(text[, pairDelim[, keyValueDelim]]) - Creates a map after splitting the text into key/value pairs using delimiters. Default delimiters are ',' for \npairDelim\n and ':' for \nkeyValueDelim\n.\n\n\nExamples:\n\n\n> SELECT str_to_map('a:1,b:2,c:3', ',', ':');\n map(\"a\":\"1\",\"b\":\"2\",\"c\":\"3\")\n> SELECT str_to_map('a');\n map(\"a\":null)\n\n\n\n\n\n\nstring\n\n\nstring(expr) - Casts the value \nexpr\n to the target data type \nstring\n.\n\n\n\n\nstruct\n\n\nstruct(col1, col2, col3, ...) - Creates a struct with the given field values.\n\n\n\n\nsubstr\n\n\nsubstr(str, pos[, len]) - Returns the substring of \nstr\n that starts at \npos\n and is of length \nlen\n, or the slice of byte array that starts at \npos\n and is of length \nlen\n.\n\n\nExamples:\n\n\n> SELECT substr('Spark SQL', 5);\n k SQL\n> SELECT substr('Spark SQL', -3);\n SQL\n> SELECT substr('Spark SQL', 5, 1);\n k\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nsubstring\n\n\nsubstring(str, pos[, len]) - Returns the substring of \nstr\n that starts at \npos\n and is of length \nlen\n, or the slice of byte array that starts at \npos\n and is of length \nlen\n.\n\n\nExamples:\n\n\n> SELECT substring('Spark SQL', 5);\n k SQL\n> SELECT substring('Spark SQL', -3);\n SQL\n> SELECT substring('Spark SQL', 5, 1);\n k\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nsubstring_index\n\n\nsubstring_index(str, delim, count) - Returns the substring from \nstr\n before \ncount\n occurrences of the delimiter \ndelim\n.\nIf \ncount\n is positive, everything to the left of the final delimiter (counting from the\nleft) is returned. If \ncount\n is negative, everything to the right of the final delimiter\n(counting from the right) is returned. The function substring_index performs a case-sensitive match\nwhen searching for \ndelim\n.\n\n\nExamples:\n\n\n> SELECT substring_index('www.apache.org', '.', 2);\n www.apache\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nsum\n\n\nsum(expr) - Returns the sum calculated from values of a group.\n\n\n\n\ntan\n\n\ntan(expr) - Returns the tangent of \nexpr\n, as if computed by \njava.lang.Math.tan\n.\n\n\nArguments:\n\n\n\n\nexpr - angle in radians\n\n\n\n\nExamples:\n\n\n> SELECT tan(0);\n 0.0\n\n\n\n\n\n\ntanh\n\n\ntanh(expr) - Returns the hyperbolic tangent of \nexpr\n, as if computed by\n\njava.lang.Math.tanh\n.\n\n\nArguments:\n\n\n\n\nexpr - hyperbolic angle\n\n\n\n\nExamples:\n\n\n> SELECT tanh(0);\n 0.0\n\n\n\n\n\n\ntimestamp\n\n\ntimestamp(expr) - Casts the value \nexpr\n to the target data type \ntimestamp\n.\n\n\n\n\ntinyint\n\n\ntinyint(expr) - Casts the value \nexpr\n to the target data type \ntinyint\n.\n\n\n\n\nto_date\n\n\nto_date(date_str[, fmt]) - Parses the \ndate_str\n expression with the \nfmt\n expression to\na date. Returns null with invalid input. By default, it follows casting rules to a date if\nthe \nfmt\n is omitted.\n\n\nExamples:\n\n\n> SELECT to_date('2009-07-30 04:17:52');\n 2009-07-30\n> SELECT to_date('2016-12-31', 'yyyy-MM-dd');\n 2016-12-31\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nto_json\n\n\nto_json(expr[, options]) - Returns a json string with a given struct value\n\n\nExamples:\n\n\n> SELECT to_json(named_struct('a', 1, 'b', 2));\n {\"a\":1,\"b\":2}\n> SELECT to_json(named_struct('time', to_timestamp('2015-08-26', 'yyyy-MM-dd')), map('timestampFormat', 'dd/MM/yyyy'));\n {\"time\":\"26/08/2015\"}\n> SELECT to_json(array(named_struct('a', 1, 'b', 2)));\n [{\"a\":1,\"b\":2}]\n> SELECT to_json(map('a', named_struct('b', 1)));\n {\"a\":{\"b\":1}}\n> SELECT to_json(map(named_struct('a', 1),named_struct('b', 2)));\n {\"[1]\":{\"b\":2}}\n> SELECT to_json(map('a', 1));\n {\"a\":1}\n> SELECT to_json(array((map('a', 1))));\n [{\"a\":1}]\n\n\n\n\nSince:\n 2.2.0\n\n\n\n\nto_timestamp\n\n\nto_timestamp(timestamp[, fmt]) - Parses the \ntimestamp\n expression with the \nfmt\n expression to\na timestamp. Returns null with invalid input. By default, it follows casting rules to\na timestamp if the \nfmt\n is omitted.\n\n\nExamples:\n\n\n> SELECT to_timestamp('2016-12-31 00:12:00');\n 2016-12-31 00:12:00\n> SELECT to_timestamp('2016-12-31', 'yyyy-MM-dd');\n 2016-12-31 00:00:00\n\n\n\n\nSince:\n 2.2.0\n\n\n\n\nto_unix_timestamp\n\n\nto_unix_timestamp(expr[, pattern]) - Returns the UNIX timestamp of the given time.\n\n\nExamples:\n\n\n> SELECT to_unix_timestamp('2016-04-08', 'yyyy-MM-dd');\n 1460041200\n\n\n\n\nSince:\n 1.6.0\n\n\n\n\nto_utc_timestamp\n\n\nto_utc_timestamp(timestamp, timezone) - Given a timestamp like '2017-07-14 02:40:00.0', interprets it as a time in the given time zone, and renders that time as a timestamp in UTC. For example, 'GMT+1' would yield '2017-07-14 01:40:00.0'.\n\n\nExamples:\n\n\n> SELECT to_utc_timestamp('2016-08-31', 'Asia/Seoul');\n 2016-08-30 15:00:00\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\ntranslate\n\n\ntranslate(input, from, to) - Translates the \ninput\n string by replacing the characters present in the \nfrom\n string with the corresponding characters in the \nto\n string.\n\n\nExamples:\n\n\n> SELECT translate('AaBbCc', 'abc', '123');\n A1B2C3\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\ntrim\n\n\ntrim(str) - Removes the leading and trailing space characters from \nstr\n.\n\n\ntrim(BOTH trimStr FROM str) - Remove the leading and trailing \ntrimStr\n characters from \nstr\n\n\ntrim(LEADING trimStr FROM str) - Remove the leading \ntrimStr\n characters from \nstr\n\n\ntrim(TRAILING trimStr FROM str) - Remove the trailing \ntrimStr\n characters from \nstr\n\n\nArguments:\n\n\n\n\nstr - a string expression\n\n\ntrimStr - the trim string characters to trim, the default value is a single space\n\n\nBOTH, FROM - these are keywords to specify trimming string characters from both ends of\n the string\n\n\nLEADING, FROM - these are keywords to specify trimming string characters from the left\n end of the string\n\n\nTRAILING, FROM - these are keywords to specify trimming string characters from the right\n end of the string\n\n\n\n\nExamples:\n\n\n> SELECT trim(' SparkSQL ');\n SparkSQL\n> SELECT trim('SL', 'SSparkSQLS');\n parkSQ\n> SELECT trim(BOTH 'SL' FROM 'SSparkSQLS');\n parkSQ\n> SELECT trim(LEADING 'SL' FROM 'SSparkSQLS');\n parkSQLS\n> SELECT trim(TRAILING 'SL' FROM 'SSparkSQLS');\n SSparkSQ\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\ntrunc\n\n\ntrunc(date, fmt) - Returns \ndate\n with the time portion of the day truncated to the unit specified by the format model \nfmt\n.\n\nfmt\n should be one of [\"year\", \"yyyy\", \"yy\", \"mon\", \"month\", \"mm\"]\n\n\nExamples:\n\n\n> SELECT trunc('2009-02-12', 'MM');\n 2009-02-01\n> SELECT trunc('2015-10-27', 'YEAR');\n 2015-01-01\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nucase\n\n\nucase(str) - Returns \nstr\n with all characters changed to uppercase.\n\n\nExamples:\n\n\n> SELECT ucase('SparkSql');\n SPARKSQL\n\n\n\n\nSince:\n 1.0.1\n\n\n\n\nunbase64\n\n\nunbase64(str) - Converts the argument from a base 64 string \nstr\n to a binary.\n\n\nExamples:\n\n\n> SELECT unbase64('U3BhcmsgU1FM');\n Spark SQL\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nunhex\n\n\nunhex(expr) - Converts hexadecimal \nexpr\n to binary.\n\n\nExamples:\n\n\n> SELECT decode(unhex('537061726B2053514C'), 'UTF-8');\n Spark SQL\n\n\n\n\n\n\nunix_timestamp\n\n\nunix_timestamp([expr[, pattern]]) - Returns the UNIX timestamp of current or specified time.\n\n\nExamples:\n\n\n> SELECT unix_timestamp();\n 1476884637\n> SELECT unix_timestamp('2016-04-08', 'yyyy-MM-dd');\n 1460041200\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nupper\n\n\nupper(str) - Returns \nstr\n with all characters changed to uppercase.\n\n\nExamples:\n\n\n> SELECT upper('SparkSql');\n SPARKSQL\n\n\n\n\nSince:\n 1.0.1\n\n\n\n\nuuid\n\n\nuuid() - Returns an universally unique identifier (UUID) string. The value is returned as a canonical UUID 36-character string.\n\n\nExamples:\n\n\n> SELECT uuid();\n 46707d92-02f4-4817-8116-a4c3b23e6266\n\n\n\n\n\n\nvar_pop\n\n\nvar_pop(expr) - Returns the population variance calculated from values of a group.\n\n\n\n\nvar_samp\n\n\nvar_samp(expr) - Returns the sample variance calculated from values of a group.\n\n\n\n\nvariance\n\n\nvariance(expr) - Returns the sample variance calculated from values of a group.\n\n\n\n\nweekofyear\n\n\nweekofyear(date) - Returns the week of the year of the given date. A week is considered to start on a Monday and week 1 is the first week with >3 days.\n\n\nExamples:\n\n\n> SELECT weekofyear('2008-02-20');\n 8\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\nwhen\n\n\nCASE WHEN expr1 THEN expr2 [WHEN expr3 THEN expr4]* [ELSE expr5] END - When \nexpr1\n = true, returns \nexpr2\n; else when \nexpr3\n = true, returns \nexpr4\n; else returns \nexpr5\n.\n\n\nArguments:\n\n\n\n\nexpr1, expr3 - the branch condition expressions should all be boolean type.\n\n\nexpr2, expr4, expr5 - the branch value expressions and else value expression should all be\n same type or coercible to a common type.\n\n\n\n\nExamples:\n\n\n> SELECT CASE WHEN 1 > 0 THEN 1 WHEN 2 > 0 THEN 2.0 ELSE 1.2 END;\n 1\n> SELECT CASE WHEN 1 < 0 THEN 1 WHEN 2 > 0 THEN 2.0 ELSE 1.2 END;\n 2\n> SELECT CASE WHEN 1 < 0 THEN 1 WHEN 2 < 0 THEN 2.0 END;\n NULL\n\n\n\n\n\n\nwindow\n\n\n\n\nxpath\n\n\nxpath(xml, xpath) - Returns a string array of values within the nodes of xml that match the XPath expression.\n\n\nExamples:\n\n\n> SELECT xpath('<a><b>b1</b><b>b2</b><b>b3</b><c>c1</c><c>c2</c></a>','a/b/text()');\n ['b1','b2','b3']\n\n\n\n\n\n\nxpath_boolean\n\n\nxpath_boolean(xml, xpath) - Returns true if the XPath expression evaluates to true, or if a matching node is found.\n\n\nExamples:\n\n\n> SELECT xpath_boolean('<a><b>1</b></a>','a/b');\n true\n\n\n\n\n\n\nxpath_double\n\n\nxpath_double(xml, xpath) - Returns a double value, the value zero if no match is found, or NaN if a match is found but the value is non-numeric.\n\n\nExamples:\n\n\n> SELECT xpath_double('<a><b>1</b><b>2</b></a>', 'sum(a/b)');\n 3.0\n\n\n\n\n\n\nxpath_float\n\n\nxpath_float(xml, xpath) - Returns a float value, the value zero if no match is found, or NaN if a match is found but the value is non-numeric.\n\n\nExamples:\n\n\n> SELECT xpath_float('<a><b>1</b><b>2</b></a>', 'sum(a/b)');\n 3.0\n\n\n\n\n\n\nxpath_int\n\n\nxpath_int(xml, xpath) - Returns an integer value, or the value zero if no match is found, or a match is found but the value is non-numeric.\n\n\nExamples:\n\n\n> SELECT xpath_int('<a><b>1</b><b>2</b></a>', 'sum(a/b)');\n 3\n\n\n\n\n\n\nxpath_long\n\n\nxpath_long(xml, xpath) - Returns a long integer value, or the value zero if no match is found, or a match is found but the value is non-numeric.\n\n\nExamples:\n\n\n> SELECT xpath_long('<a><b>1</b><b>2</b></a>', 'sum(a/b)');\n 3\n\n\n\n\n\n\nxpath_number\n\n\nxpath_number(xml, xpath) - Returns a double value, the value zero if no match is found, or NaN if a match is found but the value is non-numeric.\n\n\nExamples:\n\n\n> SELECT xpath_number('<a><b>1</b><b>2</b></a>', 'sum(a/b)');\n 3.0\n\n\n\n\n\n\nxpath_short\n\n\nxpath_short(xml, xpath) - Returns a short integer value, or the value zero if no match is found, or a match is found but the value is non-numeric.\n\n\nExamples:\n\n\n> SELECT xpath_short('<a><b>1</b><b>2</b></a>', 'sum(a/b)');\n 3\n\n\n\n\n\n\nxpath_string\n\n\nxpath_string(xml, xpath) - Returns the text contents of the first xml node that matches the XPath expression.\n\n\nExamples:\n\n\n> SELECT xpath_string('<a><b>b</b><c>cc</c></a>','a/c');\n cc\n\n\n\n\n\n\nyear\n\n\nyear(date) - Returns the year component of the date/timestamp.\n\n\nExamples:\n\n\n> SELECT year('2016-07-30');\n 2016\n\n\n\n\nSince:\n 1.5.0\n\n\n\n\n|\n\n\nexpr1 | expr2 - Returns the result of bitwise OR of \nexpr1\n and \nexpr2\n.\n\n\nExamples:\n\n\n> SELECT 3 | 5;\n 7\n\n\n\n\n\n\n~\n\n\n~ expr - Returns the result of bitwise NOT of \nexpr\n.\n\n\nExamples:\n\n\n> SELECT ~ 0;\n -1",
"title": "Functions"
},
{
"location": "/#_1",
"text": "! expr - Logical not.",
"title": "!"
},
{
"location": "/#_2",
"text": "expr1 % expr2 - Returns the remainder after expr1 / expr2 . Examples: > SELECT 2 % 1.8;\n 0.2\n> SELECT MOD(2, 1.8);\n 0.2",
"title": "%"
},
{
"location": "/#_3",
"text": "expr1 & expr2 - Returns the result of bitwise AND of expr1 and expr2 . Examples: > SELECT 3 & 5;\n 1",
"title": "&"
},
{
"location": "/#_4",
"text": "expr1 * expr2 - Returns expr1 * expr2 . Examples: > SELECT 2 * 3;\n 6",
"title": "*"
},
{
"location": "/#_5",
"text": "expr1 + expr2 - Returns expr1 + expr2 . Examples: > SELECT 1 + 2;\n 3",
"title": "+"
},
{
"location": "/#-",
"text": "expr1 - expr2 - Returns expr1 - expr2 . Examples: > SELECT 2 - 1;\n 1",
"title": "-"
},
{
"location": "/#_6",
"text": "expr1 / expr2 - Returns expr1 / expr2 . It always performs floating point division. Examples: > SELECT 3 / 2;\n 1.5\n> SELECT 2L / 2L;\n 1.0",
"title": "/"
},
{
"location": "/#_7",
"text": "expr1 < expr2 - Returns true if expr1 is less than expr2 . Arguments: expr1, expr2 - the two expressions must be same type or can be casted to a common type,\n and must be a type that can be ordered. For example, map type is not orderable, so it\n is not supported. For complex types such array/struct, the data types of fields must\n be orderable. Examples: > SELECT 1 < 2;\n true\n> SELECT 1.1 < '1';\n false\n> SELECT to_date('2009-07-30 04:17:52') < to_date('2009-07-30 04:17:52');\n false\n> SELECT to_date('2009-07-30 04:17:52') < to_date('2009-08-01 04:17:52');\n true\n> SELECT 1 < NULL;\n NULL",
"title": "<"
},
{
"location": "/#_8",
"text": "expr1 <= expr2 - Returns true if expr1 is less than or equal to expr2 . Arguments: expr1, expr2 - the two expressions must be same type or can be casted to a common type,\n and must be a type that can be ordered. For example, map type is not orderable, so it\n is not supported. For complex types such array/struct, the data types of fields must\n be orderable. Examples: > SELECT 2 <= 2;\n true\n> SELECT 1.0 <= '1';\n true\n> SELECT to_date('2009-07-30 04:17:52') <= to_date('2009-07-30 04:17:52');\n true\n> SELECT to_date('2009-07-30 04:17:52') <= to_date('2009-08-01 04:17:52');\n true\n> SELECT 1 <= NULL;\n NULL",
"title": "<="
},
{
"location": "/#_9",
"text": "expr1 <=> expr2 - Returns same result as the EQUAL(=) operator for non-null operands,\nbut returns true if both are null, false if one of the them is null. Arguments: expr1, expr2 - the two expressions must be same type or can be casted to a common type,\n and must be a type that can be used in equality comparison. Map type is not supported.\n For complex types such array/struct, the data types of fields must be orderable. Examples: > SELECT 2 <=> 2;\n true\n> SELECT 1 <=> '1';\n true\n> SELECT true <=> NULL;\n false\n> SELECT NULL <=> NULL;\n true",
"title": "<=>"
},
{
"location": "/#_10",
"text": "expr1 = expr2 - Returns true if expr1 equals expr2 , or false otherwise. Arguments: expr1, expr2 - the two expressions must be same type or can be casted to a common type,\n and must be a type that can be used in equality comparison. Map type is not supported.\n For complex types such array/struct, the data types of fields must be orderable. Examples: > SELECT 2 = 2;\n true\n> SELECT 1 = '1';\n true\n> SELECT true = NULL;\n NULL\n> SELECT NULL = NULL;\n NULL",
"title": "="
},
{
"location": "/#_11",
"text": "expr1 == expr2 - Returns true if expr1 equals expr2 , or false otherwise. Arguments: expr1, expr2 - the two expressions must be same type or can be casted to a common type,\n and must be a type that can be used in equality comparison. Map type is not supported.\n For complex types such array/struct, the data types of fields must be orderable. Examples: > SELECT 2 == 2;\n true\n> SELECT 1 == '1';\n true\n> SELECT true == NULL;\n NULL\n> SELECT NULL == NULL;\n NULL",
"title": "=="
},
{
"location": "/#_12",
"text": "expr1 > expr2 - Returns true if expr1 is greater than expr2 . Arguments: expr1, expr2 - the two expressions must be same type or can be casted to a common type,\n and must be a type that can be ordered. For example, map type is not orderable, so it\n is not supported. For complex types such array/struct, the data types of fields must\n be orderable. Examples: > SELECT 2 > 1;\n true\n> SELECT 2 > '1.1';\n true\n> SELECT to_date('2009-07-30 04:17:52') > to_date('2009-07-30 04:17:52');\n false\n> SELECT to_date('2009-07-30 04:17:52') > to_date('2009-08-01 04:17:52');\n false\n> SELECT 1 > NULL;\n NULL",
"title": ">"
},
{
"location": "/#_13",
"text": "expr1 >= expr2 - Returns true if expr1 is greater than or equal to expr2 . Arguments: expr1, expr2 - the two expressions must be same type or can be casted to a common type,\n and must be a type that can be ordered. For example, map type is not orderable, so it\n is not supported. For complex types such array/struct, the data types of fields must\n be orderable. Examples: > SELECT 2 >= 1;\n true\n> SELECT 2.0 >= '2.1';\n false\n> SELECT to_date('2009-07-30 04:17:52') >= to_date('2009-07-30 04:17:52');\n true\n> SELECT to_date('2009-07-30 04:17:52') >= to_date('2009-08-01 04:17:52');\n false\n> SELECT 1 >= NULL;\n NULL",
"title": ">="
},
{
"location": "/#_14",
"text": "expr1 ^ expr2 - Returns the result of bitwise exclusive OR of expr1 and expr2 . Examples: > SELECT 3 ^ 5;\n 6",
"title": "^"
},
{
"location": "/#abs",
"text": "abs(expr) - Returns the absolute value of the numeric value. Examples: > SELECT abs(-1);\n 1",
"title": "abs"
},
{
"location": "/#acos",
"text": "acos(expr) - Returns the inverse cosine (a.k.a. arc cosine) of expr , as if computed by java.lang.Math.acos . Examples: > SELECT acos(1);\n 0.0\n> SELECT acos(2);\n NaN",
"title": "acos"
},
{
"location": "/#add_months",
"text": "add_months(start_date, num_months) - Returns the date that is num_months after start_date . Examples: > SELECT add_months('2016-08-31', 1);\n 2016-09-30 Since: 1.5.0",
"title": "add_months"
},
{
"location": "/#and",
"text": "expr1 and expr2 - Logical AND.",
"title": "and"
},
{
"location": "/#approx_count_distinct",
"text": "approx_count_distinct(expr[, relativeSD]) - Returns the estimated cardinality by HyperLogLog++. relativeSD defines the maximum estimation error allowed.",
"title": "approx_count_distinct"
},
{
"location": "/#approx_percentile",
"text": "approx_percentile(col, percentage [, accuracy]) - Returns the approximate percentile value of numeric\ncolumn col at the given percentage. The value of percentage must be between 0.0\nand 1.0. The accuracy parameter (default: 10000) is a positive numeric literal which\ncontrols approximation accuracy at the cost of memory. Higher value of accuracy yields\nbetter accuracy, 1.0/accuracy is the relative error of the approximation.\nWhen percentage is an array, each value of the percentage array must be between 0.0 and 1.0.\nIn this case, returns the approximate percentile array of column col at the given\npercentage array. Examples: > SELECT approx_percentile(10.0, array(0.5, 0.4, 0.1), 100);\n [10.0,10.0,10.0]\n> SELECT approx_percentile(10.0, 0.5, 100);\n 10.0",
"title": "approx_percentile"
},
{
"location": "/#array",
"text": "array(expr, ...) - Returns an array with the given elements. Examples: > SELECT array(1, 2, 3);\n [1,2,3]",
"title": "array"
},
{
"location": "/#array_contains",
"text": "array_contains(array, value) - Returns true if the array contains the value. Examples: > SELECT array_contains(array(1, 2, 3), 2);\n true",
"title": "array_contains"
},
{
"location": "/#ascii",
"text": "ascii(str) - Returns the numeric value of the first character of str . Examples: > SELECT ascii('222');\n 50\n> SELECT ascii(2);\n 50 Since: 1.5.0",
"title": "ascii"
},
{
"location": "/#asin",
"text": "asin(expr) - Returns the inverse sine (a.k.a. arc sine) the arc sin of expr ,\nas if computed by java.lang.Math.asin . Examples: > SELECT asin(0);\n 0.0\n> SELECT asin(2);\n NaN",
"title": "asin"
},
{
"location": "/#assert_true",
"text": "assert_true(expr) - Throws an exception if expr is not true. Examples: > SELECT assert_true(0 < 1);\n NULL",
"title": "assert_true"
},
{
"location": "/#atan",
"text": "atan(expr) - Returns the inverse tangent (a.k.a. arc tangent) of expr , as if computed by java.lang.Math.atan Examples: > SELECT atan(0);\n 0.0",
"title": "atan"
},
{
"location": "/#atan2",
"text": "atan2(exprY, exprX) - Returns the angle in radians between the positive x-axis of a plane\nand the point given by the coordinates ( exprX , exprY ), as if computed by java.lang.Math.atan2 . Arguments: exprY - coordinate on y-axis exprX - coordinate on x-axis Examples: > SELECT atan2(0, 0);\n 0.0",
"title": "atan2"
},
{
"location": "/#avg",
"text": "avg(expr) - Returns the mean calculated from values of a group.",
"title": "avg"
},
{
"location": "/#base64",
"text": "base64(bin) - Converts the argument from a binary bin to a base 64 string. Examples: > SELECT base64('Spark SQL');\n U3BhcmsgU1FM Since: 1.5.0",
"title": "base64"
},
{
"location": "/#bigint",
"text": "bigint(expr) - Casts the value expr to the target data type bigint .",
"title": "bigint"
},
{
"location": "/#bin",
"text": "bin(expr) - Returns the string representation of the long value expr represented in binary. Examples: > SELECT bin(13);\n 1101\n> SELECT bin(-13);\n 1111111111111111111111111111111111111111111111111111111111110011\n> SELECT bin(13.3);\n 1101",
"title": "bin"
},
{
"location": "/#binary",
"text": "binary(expr) - Casts the value expr to the target data type binary .",
"title": "binary"
},
{
"location": "/#bit_length",
"text": "bit_length(expr) - Returns the bit length of string data or number of bits of binary data. Examples: > SELECT bit_length('Spark SQL');\n 72 Since: 2.3.0",
"title": "bit_length"
},
{
"location": "/#boolean",
"text": "boolean(expr) - Casts the value expr to the target data type boolean .",
"title": "boolean"
},
{
"location": "/#bround",
"text": "bround(expr, d) - Returns expr rounded to d decimal places using HALF_EVEN rounding mode. Examples: > SELECT bround(2.5, 0);\n 2.0",
"title": "bround"
},
{
"location": "/#cast",
"text": "cast(expr AS type) - Casts the value expr to the target data type type . Examples: > SELECT cast('10' as int);\n 10",
"title": "cast"
},
{
"location": "/#cbrt",
"text": "cbrt(expr) - Returns the cube root of expr . Examples: > SELECT cbrt(27.0);\n 3.0",
"title": "cbrt"
},
{
"location": "/#ceil",
"text": "ceil(expr) - Returns the smallest integer not smaller than expr . Examples: > SELECT ceil(-0.1);\n 0\n> SELECT ceil(5);\n 5",
"title": "ceil"
},
{
"location": "/#ceiling",
"text": "ceiling(expr) - Returns the smallest integer not smaller than expr . Examples: > SELECT ceiling(-0.1);\n 0\n> SELECT ceiling(5);\n 5",
"title": "ceiling"
},
{
"location": "/#char",
"text": "char(expr) - Returns the ASCII character having the binary equivalent to expr . If n is larger than 256 the result is equivalent to chr(n % 256) Examples: > SELECT char(65);\n A Since: 2.3.0",
"title": "char"
},
{
"location": "/#char_length",
"text": "char_length(expr) - Returns the character length of string data or number of bytes of binary data. The length of string data includes the trailing spaces. The length of binary data includes binary zeros. Examples: > SELECT char_length('Spark SQL ');\n 10\n> SELECT CHAR_LENGTH('Spark SQL ');\n 10\n> SELECT CHARACTER_LENGTH('Spark SQL ');\n 10 Since: 1.5.0",
"title": "char_length"
},
{
"location": "/#character_length",
"text": "character_length(expr) - Returns the character length of string data or number of bytes of binary data. The length of string data includes the trailing spaces. The length of binary data includes binary zeros. Examples: > SELECT character_length('Spark SQL ');\n 10\n> SELECT CHAR_LENGTH('Spark SQL ');\n 10\n> SELECT CHARACTER_LENGTH('Spark SQL ');\n 10 Since: 1.5.0",
"title": "character_length"
},
{
"location": "/#chr",
"text": "chr(expr) - Returns the ASCII character having the binary equivalent to expr . If n is larger than 256 the result is equivalent to chr(n % 256) Examples: > SELECT chr(65);\n A Since: 2.3.0",
"title": "chr"
},
{
"location": "/#coalesce",
"text": "coalesce(expr1, expr2, ...) - Returns the first non-null argument if exists. Otherwise, null. Examples: > SELECT coalesce(NULL, 1, NULL);\n 1 Since: 1.0.0",
"title": "coalesce"
},
{
"location": "/#collect_list",
"text": "collect_list(expr) - Collects and returns a list of non-unique elements.",
"title": "collect_list"
},
{
"location": "/#collect_set",
"text": "collect_set(expr) - Collects and returns a set of unique elements.",
"title": "collect_set"
},
{
"location": "/#concat",
"text": "concat(str1, str2, ..., strN) - Returns the concatenation of str1, str2, ..., strN. Examples: > SELECT concat('Spark', 'SQL');\n SparkSQL",
"title": "concat"
},
{
"location": "/#concat_ws",
"text": "concat_ws(sep, [str | array(str)]+) - Returns the concatenation of the strings separated by sep . Examples: > SELECT concat_ws(' ', 'Spark', 'SQL');\n Spark SQL Since: 1.5.0",
"title": "concat_ws"
},
{
"location": "/#conv",
"text": "conv(num, from_base, to_base) - Convert num from from_base to to_base . Examples: > SELECT conv('100', 2, 10);\n 4\n> SELECT conv(-10, 16, -10);\n -16",
"title": "conv"
},
{
"location": "/#corr",
"text": "corr(expr1, expr2) - Returns Pearson coefficient of correlation between a set of number pairs.",
"title": "corr"
},
{
"location": "/#cos",
"text": "cos(expr) - Returns the cosine of expr , as if computed by java.lang.Math.cos . Arguments: expr - angle in radians Examples: > SELECT cos(0);\n 1.0",
"title": "cos"
},
{
"location": "/#cosh",
"text": "cosh(expr) - Returns the hyperbolic cosine of expr , as if computed by java.lang.Math.cosh . Arguments: expr - hyperbolic angle Examples: > SELECT cosh(0);\n 1.0",
"title": "cosh"
},
{
"location": "/#cot",
"text": "cot(expr) - Returns the cotangent of expr , as if computed by 1/java.lang.Math.cot . Arguments: expr - angle in radians Examples: > SELECT cot(1);\n 0.6420926159343306",
"title": "cot"
},
{
"location": "/#count",
"text": "count(*) - Returns the total number of retrieved rows, including rows containing null. count(expr) - Returns the number of rows for which the supplied expression is non-null. count(DISTINCT expr[, expr...]) - Returns the number of rows for which the supplied expression(s) are unique and non-null.",
"title": "count"
},
{
"location": "/#count_min_sketch",
"text": "count_min_sketch(col, eps, confidence, seed) - Returns a count-min sketch of a column with the given esp,\nconfidence and seed. The result is an array of bytes, which can be deserialized to a CountMinSketch before usage. Count-min sketch is a probabilistic data structure used for\ncardinality estimation using sub-linear space.",
"title": "count_min_sketch"
},
{
"location": "/#covar_pop",
"text": "covar_pop(expr1, expr2) - Returns the population covariance of a set of number pairs.",
"title": "covar_pop"
},
{
"location": "/#covar_samp",
"text": "covar_samp(expr1, expr2) - Returns the sample covariance of a set of number pairs.",
"title": "covar_samp"
},
{
"location": "/#crc32",
"text": "crc32(expr) - Returns a cyclic redundancy check value of the expr as a bigint. Examples: > SELECT crc32('Spark');\n 1557323817",
"title": "crc32"
},
{
"location": "/#cube",
"text": "cube([col1[, col2 ..]]) - create a multi-dimensional cube using the specified columns\nso that we can run aggregation on them. Examples: > SELECT name, age, count(*) FROM VALUES (2, 'Alice'), (5, 'Bob') people(age, name) GROUP BY cube(name, age);\n NULL 2 1\n NULL NULL 2\n Alice 2 1\n Bob 5 1\n NULL 5 1\n Bob NULL 1\n Alice NULL 1 Since: 2.0.0",
"title": "cube"
},
{
"location": "/#cume_dist",
"text": "cume_dist() - Computes the position of a value relative to all values in the partition.",
"title": "cume_dist"
},
{
"location": "/#current_database",
"text": "current_database() - Returns the current database. Examples: > SELECT current_database();\n default",
"title": "current_database"
},
{
"location": "/#current_date",
"text": "current_date() - Returns the current date at the start of query evaluation. Since: 1.5.0",
"title": "current_date"
},
{
"location": "/#current_timestamp",
"text": "current_timestamp() - Returns the current timestamp at the start of query evaluation. Since: 1.5.0",
"title": "current_timestamp"
},
{
"location": "/#date",
"text": "date(expr) - Casts the value expr to the target data type date .",
"title": "date"
},
{
"location": "/#date_add",
"text": "date_add(start_date, num_days) - Returns the date that is num_days after start_date . Examples: > SELECT date_add('2016-07-30', 1);\n 2016-07-31 Since: 1.5.0",
"title": "date_add"
},
{
"location": "/#date_format",
"text": "date_format(timestamp, fmt) - Converts timestamp to a value of string in the format specified by the date format fmt . Examples: > SELECT date_format('2016-04-08', 'y');\n 2016 Since: 1.5.0",
"title": "date_format"
},
{
"location": "/#date_sub",
"text": "date_sub(start_date, num_days) - Returns the date that is num_days before start_date . Examples: > SELECT date_sub('2016-07-30', 1);\n 2016-07-29 Since: 1.5.0",
"title": "date_sub"
},
{
"location": "/#date_trunc",
"text": "date_trunc(fmt, ts) - Returns timestamp ts truncated to the unit specified by the format model fmt . fmt should be one of [\"YEAR\", \"YYYY\", \"YY\", \"MON\", \"MONTH\", \"MM\", \"DAY\", \"DD\", \"HOUR\", \"MINUTE\", \"SECOND\", \"WEEK\", \"QUARTER\"] Examples: > SELECT date_trunc('YEAR', '2015-03-05T09:32:05.359');\n 2015-01-01 00:00:00\n> SELECT date_trunc('MM', '2015-03-05T09:32:05.359');\n 2015-03-01 00:00:00\n> SELECT date_trunc('DD', '2015-03-05T09:32:05.359');\n 2015-03-05 00:00:00\n> SELECT date_trunc('HOUR', '2015-03-05T09:32:05.359');\n 2015-03-05 09:00:00 Since: 2.3.0",
"title": "date_trunc"
},
{
"location": "/#datediff",
"text": "datediff(endDate, startDate) - Returns the number of days from startDate to endDate . Examples: > SELECT datediff('2009-07-31', '2009-07-30');\n 1\n\n> SELECT datediff('2009-07-30', '2009-07-31');\n -1 Since: 1.5.0",
"title": "datediff"
},
{
"location": "/#day",
"text": "day(date) - Returns the day of month of the date/timestamp. Examples: > SELECT day('2009-07-30');\n 30 Since: 1.5.0",
"title": "day"
},
{
"location": "/#dayofmonth",
"text": "dayofmonth(date) - Returns the day of month of the date/timestamp. Examples: > SELECT dayofmonth('2009-07-30');\n 30 Since: 1.5.0",
"title": "dayofmonth"
},
{
"location": "/#dayofweek",
"text": "dayofweek(date) - Returns the day of the week for date/timestamp (1 = Sunday, 2 = Monday, ..., 7 = Saturday). Examples: > SELECT dayofweek('2009-07-30');\n 5 Since: 2.3.0",
"title": "dayofweek"
},
{
"location": "/#dayofyear",
"text": "dayofyear(date) - Returns the day of year of the date/timestamp. Examples: > SELECT dayofyear('2016-04-09');\n 100 Since: 1.5.0",
"title": "dayofyear"
},
{
"location": "/#decimal",
"text": "decimal(expr) - Casts the value expr to the target data type decimal .",
"title": "decimal"
},
{
"location": "/#decode",
"text": "decode(bin, charset) - Decodes the first argument using the second argument character set. Examples: > SELECT decode(encode('abc', 'utf-8'), 'utf-8');\n abc Since: 1.5.0",
"title": "decode"
},
{
"location": "/#degrees",
"text": "degrees(expr) - Converts radians to degrees. Arguments: expr - angle in radians Examples: > SELECT degrees(3.141592653589793);\n 180.0",
"title": "degrees"
},
{
"location": "/#dense_rank",
"text": "dense_rank() - Computes the rank of a value in a group of values. The result is one plus the\npreviously assigned rank value. Unlike the function rank, dense_rank will not produce gaps\nin the ranking sequence.",
"title": "dense_rank"
},
{
"location": "/#double",
"text": "double(expr) - Casts the value expr to the target data type double .",
"title": "double"
},
{
"location": "/#e",
"text": "e() - Returns Euler's number, e. Examples: > SELECT e();\n 2.718281828459045",
"title": "e"
},
{
"location": "/#elt",
"text": "elt(n, input1, input2, ...) - Returns the n -th input, e.g., returns input2 when n is 2. Examples: > SELECT elt(1, 'scala', 'java');\n scala Since: 2.0.0",
"title": "elt"
},
{
"location": "/#encode",
"text": "encode(str, charset) - Encodes the first argument using the second argument character set. Examples: > SELECT encode('abc', 'utf-8');\n abc Since: 1.5.0",
"title": "encode"
},
{
"location": "/#exp",
"text": "exp(expr) - Returns e to the power of expr . Examples: > SELECT exp(0);\n 1.0",
"title": "exp"
},
{
"location": "/#explode",
"text": "explode(expr) - Separates the elements of array expr into multiple rows, or the elements of map expr into multiple rows and columns. Examples: > SELECT explode(array(10, 20));\n 10\n 20",
"title": "explode"
},
{
"location": "/#explode_outer",
"text": "explode_outer(expr) - Separates the elements of array expr into multiple rows, or the elements of map expr into multiple rows and columns. Examples: > SELECT explode_outer(array(10, 20));\n 10\n 20",
"title": "explode_outer"
},
{
"location": "/#expm1",
"text": "expm1(expr) - Returns exp( expr ) - 1. Examples: > SELECT expm1(0);\n 0.0",
"title": "expm1"
},
{
"location": "/#factorial",
"text": "factorial(expr) - Returns the factorial of expr . expr is [0..20]. Otherwise, null. Examples: > SELECT factorial(5);\n 120",
"title": "factorial"
},
{
"location": "/#find_in_set",
"text": "find_in_set(str, str_array) - Returns the index (1-based) of the given string ( str ) in the comma-delimited list ( str_array ).\nReturns 0, if the string was not found or if the given string ( str ) contains a comma. Examples: > SELECT find_in_set('ab','abc,b,ab,c,def');\n 3 Since: 1.5.0",
"title": "find_in_set"
},
{
"location": "/#first",
"text": "first(expr[, isIgnoreNull]) - Returns the first value of expr for a group of rows.\nIf isIgnoreNull is true, returns only non-null values.",
"title": "first"
},
{
"location": "/#first_value",
"text": "first_value(expr[, isIgnoreNull]) - Returns the first value of expr for a group of rows.\nIf isIgnoreNull is true, returns only non-null values.",
"title": "first_value"
},
{
"location": "/#float",
"text": "float(expr) - Casts the value expr to the target data type float .",
"title": "float"
},
{
"location": "/#floor",
"text": "floor(expr) - Returns the largest integer not greater than expr . Examples: > SELECT floor(-0.1);\n -1\n> SELECT floor(5);\n 5",
"title": "floor"
},
{
"location": "/#format_number",
"text": "format_number(expr1, expr2) - Formats the number expr1 like '#,###,###.##', rounded to expr2 \ndecimal places. If expr2 is 0, the result has no decimal point or fractional part.\nThis is supposed to function like MySQL's FORMAT. Examples: > SELECT format_number(12332.123456, 4);\n 12,332.1235 Since: 1.5.0",
"title": "format_number"
},
{
"location": "/#format_string",
"text": "format_string(strfmt, obj, ...) - Returns a formatted string from printf-style format strings. Examples: > SELECT format_string(\"Hello World %d %s\", 100, \"days\");\n Hello World 100 days Since: 1.5.0",
"title": "format_string"
},
{
"location": "/#from_json",
"text": "from_json(jsonStr, schema[, options]) - Returns a struct value with the given jsonStr and schema . Examples: > SELECT from_json('{\"a\":1, \"b\":0.8}', 'a INT, b DOUBLE');\n {\"a\":1, \"b\":0.8}\n> SELECT from_json('{\"time\":\"26/08/2015\"}', 'time Timestamp', map('timestampFormat', 'dd/MM/yyyy'));\n {\"time\":\"2015-08-26 00:00:00.0\"} Since: 2.2.0",
"title": "from_json"
},
{
"location": "/#from_unixtime",
"text": "from_unixtime(unix_time, format) - Returns unix_time in the specified format . Examples: > SELECT from_unixtime(0, 'yyyy-MM-dd HH:mm:ss');\n 1970-01-01 00:00:00 Since: 1.5.0",
"title": "from_unixtime"
},
{
"location": "/#from_utc_timestamp",
"text": "from_utc_timestamp(timestamp, timezone) - Given a timestamp like '2017-07-14 02:40:00.0', interprets it as a time in UTC, and renders that time as a timestamp in the given time zone. For example, 'GMT+1' would yield '2017-07-14 03:40:00.0'. Examples: > SELECT from_utc_timestamp('2016-08-31', 'Asia/Seoul');\n 2016-08-31 09:00:00 Since: 1.5.0",
"title": "from_utc_timestamp"
},
{
"location": "/#get_json_object",
"text": "get_json_object(json_txt, path) - Extracts a json object from path . Examples: > SELECT get_json_object('{\"a\":\"b\"}', '$.a');\n b",
"title": "get_json_object"
},
{
"location": "/#greatest",
"text": "greatest(expr, ...) - Returns the greatest value of all parameters, skipping null values. Examples: > SELECT greatest(10, 9, 2, 4, 3);\n 10",
"title": "greatest"
},
{
"location": "/#grouping",
"text": "grouping(col) - indicates whether a specified column in a GROUP BY is aggregated or\nnot, returns 1 for aggregated or 0 for not aggregated in the result set.\", Examples: > SELECT name, grouping(name), sum(age) FROM VALUES (2, 'Alice'), (5, 'Bob') people(age, name) GROUP BY cube(name);\n Alice 0 2\n NULL 1 7\n Bob 0 5 Since: 2.0.0",
"title": "grouping"
},
{
"location": "/#grouping_id",
"text": "grouping_id([col1[, col2 ..]]) - returns the level of grouping, equals to (grouping(c1) << (n-1)) + (grouping(c2) << (n-2)) + ... + grouping(cn) Examples: > SELECT name, grouping_id(), sum(age), avg(height) FROM VALUES (2, 'Alice', 165), (5, 'Bob', 180) people(age, name, height) GROUP BY cube(name, height);\n NULL 2 2 165.0\n Alice 0 2 165.0\n NULL 2 5 180.0\n NULL 3 7 172.5\n Bob 0 5 180.0\n Bob 1 5 180.0\n Alice 1 2 165.0 Note: Input columns should match with grouping columns exactly, or empty (means all the grouping\ncolumns). Since: 2.0.0",
"title": "grouping_id"
},
{
"location": "/#hash",
"text": "hash(expr1, expr2, ...) - Returns a hash value of the arguments. Examples: > SELECT hash('Spark', array(123), 2);\n -1321691492",
"title": "hash"
},
{
"location": "/#hex",
"text": "hex(expr) - Converts expr to hexadecimal. Examples: > SELECT hex(17);\n 11\n> SELECT hex('Spark SQL');\n 537061726B2053514C",
"title": "hex"
},
{
"location": "/#hour",
"text": "hour(timestamp) - Returns the hour component of the string/timestamp. Examples: > SELECT hour('2009-07-30 12:58:59');\n 12 Since: 1.5.0",
"title": "hour"
},
{
"location": "/#hypot",
"text": "hypot(expr1, expr2) - Returns sqrt( expr1 2 + expr2 2). Examples: > SELECT hypot(3, 4);\n 5.0",
"title": "hypot"
},
{
"location": "/#if",
"text": "if(expr1, expr2, expr3) - If expr1 evaluates to true, then returns expr2 ; otherwise returns expr3 . Examples: > SELECT if(1 < 2, 'a', 'b');\n a",
"title": "if"
},
{
"location": "/#ifnull",
"text": "ifnull(expr1, expr2) - Returns expr2 if expr1 is null, or expr1 otherwise. Examples: > SELECT ifnull(NULL, array('2'));\n [\"2\"] Since: 2.0.0",
"title": "ifnull"
},
{
"location": "/#in",
"text": "expr1 in(expr2, expr3, ...) - Returns true if expr equals to any valN. Arguments: expr1, expr2, expr3, ... - the arguments must be same type. Examples: > SELECT 1 in(1, 2, 3);\n true\n> SELECT 1 in(2, 3, 4);\n false\n> SELECT named_struct('a', 1, 'b', 2) in(named_struct('a', 1, 'b', 1), named_struct('a', 1, 'b', 3));\n false\n> SELECT named_struct('a', 1, 'b', 2) in(named_struct('a', 1, 'b', 2), named_struct('a', 1, 'b', 3));\n true",
"title": "in"
},
{
"location": "/#initcap",
"text": "initcap(str) - Returns str with the first letter of each word in uppercase.\nAll other letters are in lowercase. Words are delimited by white space. Examples: > SELECT initcap('sPark sql');\n Spark Sql Since: 1.5.0",
"title": "initcap"
},
{
"location": "/#inline",
"text": "inline(expr) - Explodes an array of structs into a table. Examples: > SELECT inline(array(struct(1, 'a'), struct(2, 'b')));\n 1 a\n 2 b",
"title": "inline"
},
{
"location": "/#inline_outer",
"text": "inline_outer(expr) - Explodes an array of structs into a table. Examples: > SELECT inline_outer(array(struct(1, 'a'), struct(2, 'b')));\n 1 a\n 2 b",
"title": "inline_outer"
},
{
"location": "/#input_file_block_length",
"text": "input_file_block_length() - Returns the length of the block being read, or -1 if not available.",
"title": "input_file_block_length"
},
{
"location": "/#input_file_block_start",
"text": "input_file_block_start() - Returns the start offset of the block being read, or -1 if not available.",
"title": "input_file_block_start"
},
{
"location": "/#input_file_name",
"text": "input_file_name() - Returns the name of the file being read, or empty string if not available.",
"title": "input_file_name"
},
{
"location": "/#instr",
"text": "instr(str, substr) - Returns the (1-based) index of the first occurrence of substr in str . Examples: > SELECT instr('SparkSQL', 'SQL');\n 6 Since: 1.5.0",
"title": "instr"
},
{
"location": "/#int",
"text": "int(expr) - Casts the value expr to the target data type int .",
"title": "int"
},
{
"location": "/#isnan",
"text": "isnan(expr) - Returns true if expr is NaN, or false otherwise. Examples: > SELECT isnan(cast('NaN' as double));\n true Since: 1.5.0",
"title": "isnan"
},
{
"location": "/#isnotnull",
"text": "isnotnull(expr) - Returns true if expr is not null, or false otherwise. Examples: > SELECT isnotnull(1);\n true Since: 1.0.0",
"title": "isnotnull"
},
{
"location": "/#isnull",
"text": "isnull(expr) - Returns true if expr is null, or false otherwise. Examples: > SELECT isnull(1);\n false Since: 1.0.0",
"title": "isnull"
},
{
"location": "/#java_method",
"text": "java_method(class, method[, arg1[, arg2 ..]]) - Calls a method with reflection. Examples: > SELECT java_method('java.util.UUID', 'randomUUID');\n c33fb387-8500-4bfa-81d2-6e0e3e930df2\n> SELECT java_method('java.util.UUID', 'fromString', 'a5cf6c42-0c85-418f-af6c-3e4e5b1328f2');\n a5cf6c42-0c85-418f-af6c-3e4e5b1328f2",
"title": "java_method"
},
{
"location": "/#json_tuple",
"text": "json_tuple(jsonStr, p1, p2, ..., pn) - Returns a tuple like the function get_json_object, but it takes multiple names. All the input parameters and output column types are string. Examples: > SELECT json_tuple('{\"a\":1, \"b\":2}', 'a', 'b');\n 1 2",
"title": "json_tuple"
},
{
"location": "/#kurtosis",
"text": "kurtosis(expr) - Returns the kurtosis value calculated from values of a group.",
"title": "kurtosis"
},
{
"location": "/#lag",
"text": "lag(input[, offset[, default]]) - Returns the value of input at the offset th row\nbefore the current row in the window. The default value of offset is 1 and the default\nvalue of default is null. If the value of input at the offset th row is null,\nnull is returned. If there is no such offset row (e.g., when the offset is 1, the first\nrow of the window does not have any previous row), default is returned.",
"title": "lag"
},
{
"location": "/#last",
"text": "last(expr[, isIgnoreNull]) - Returns the last value of expr for a group of rows.\nIf isIgnoreNull is true, returns only non-null values.",
"title": "last"
},
{
"location": "/#last_day",
"text": "last_day(date) - Returns the last day of the month which the date belongs to. Examples: > SELECT last_day('2009-01-12');\n 2009-01-31 Since: 1.5.0",
"title": "last_day"
},
{
"location": "/#last_value",
"text": "last_value(expr[, isIgnoreNull]) - Returns the last value of expr for a group of rows.\nIf isIgnoreNull is true, returns only non-null values.",
"title": "last_value"
},
{
"location": "/#lcase",
"text": "lcase(str) - Returns str with all characters changed to lowercase. Examples: > SELECT lcase('SparkSql');\n sparksql Since: 1.0.1",
"title": "lcase"
},
{
"location": "/#lead",
"text": "lead(input[, offset[, default]]) - Returns the value of input at the offset th row\nafter the current row in the window. The default value of offset is 1 and the default\nvalue of default is null. If the value of input at the offset th row is null,\nnull is returned. If there is no such an offset row (e.g., when the offset is 1, the last\nrow of the window does not have any subsequent row), default is returned.",
"title": "lead"
},
{
"location": "/#least",
"text": "least(expr, ...) - Returns the least value of all parameters, skipping null values. Examples: > SELECT least(10, 9, 2, 4, 3);\n 2",
"title": "least"
},
{
"location": "/#left",
"text": "left(str, len) - Returns the leftmost len ( len can be string type) characters from the string str ,if len is less or equal than 0 the result is an empty string. Examples: > SELECT left('Spark SQL', 3);\n Spa Since: 2.3.0",
"title": "left"
},
{
"location": "/#length",
"text": "length(expr) - Returns the character length of string data or number of bytes of binary data. The length of string data includes the trailing spaces. The length of binary data includes binary zeros. Examples: > SELECT length('Spark SQL ');\n 10\n> SELECT CHAR_LENGTH('Spark SQL ');\n 10\n> SELECT CHARACTER_LENGTH('Spark SQL ');\n 10 Since: 1.5.0",
"title": "length"
},
{
"location": "/#levenshtein",
"text": "levenshtein(str1, str2) - Returns the Levenshtein distance between the two given strings. Examples: > SELECT levenshtein('kitten', 'sitting');\n 3 Since: 1.5.0",
"title": "levenshtein"
},
{
"location": "/#like",
"text": "str like pattern - Returns true if str matches pattern, null if any arguments are null, false otherwise. Arguments: str - a string expression pattern - a string expression. The pattern is a string which is matched literally, with\n exception to the following special symbols: _ matches any one character in the input (similar to . in posix regular expressions) % matches zero or more characters in the input (similar to .* in posix regular\nexpressions) The escape character is '\\'. If an escape character precedes a special symbol or another\nescape character, the following character is matched literally. It is invalid to escape\nany other character. Since Spark 2.0, string literals are unescaped in our SQL parser. For example, in order\nto match \"\\abc\", the pattern should be \"\\abc\". When SQL config 'spark.sql.parser.escapedStringLiterals' is enabled, it fallbacks\nto Spark 1.6 behavior regarding string literal parsing. For example, if the config is\nenabled, the pattern to match \"\\abc\" should be \"\\abc\". Examples: > SELECT '%SystemDrive%\\Users\\John' like '\\%SystemDrive\\%\\\\Users%'\ntrue Note: Use RLIKE to match with standard regular expressions. Since: 1.0.0",
"title": "like"
},
{
"location": "/#ln",
"text": "ln(expr) - Returns the natural logarithm (base e) of expr . Examples: > SELECT ln(1);\n 0.0",
"title": "ln"
},
{
"location": "/#locate",
"text": "locate(substr, str[, pos]) - Returns the position of the first occurrence of substr in str after position pos .\nThe given pos and return value are 1-based. Examples: > SELECT locate('bar', 'foobarbar');\n 4\n> SELECT locate('bar', 'foobarbar', 5);\n 7\n> SELECT POSITION('bar' IN 'foobarbar');\n 4 Since: 1.5.0",
"title": "locate"
},
{
"location": "/#log",
"text": "log(base, expr) - Returns the logarithm of expr with base . Examples: > SELECT log(10, 100);\n 2.0",
"title": "log"
},
{
"location": "/#log10",
"text": "log10(expr) - Returns the logarithm of expr with base 10. Examples: > SELECT log10(10);\n 1.0",
"title": "log10"
},
{
"location": "/#log1p",
"text": "log1p(expr) - Returns log(1 + expr ). Examples: > SELECT log1p(0);\n 0.0",
"title": "log1p"
},
{
"location": "/#log2",
"text": "log2(expr) - Returns the logarithm of expr with base 2. Examples: > SELECT log2(2);\n 1.0",
"title": "log2"
},
{
"location": "/#lower",
"text": "lower(str) - Returns str with all characters changed to lowercase. Examples: > SELECT lower('SparkSql');\n sparksql Since: 1.0.1",
"title": "lower"
},
{
"location": "/#lpad",
"text": "lpad(str, len, pad) - Returns str , left-padded with pad to a length of len .\nIf str is longer than len , the return value is shortened to len characters. Examples: > SELECT lpad('hi', 5, '??');\n ???hi\n> SELECT lpad('hi', 1, '??');\n h Since: 1.5.0",
"title": "lpad"
},
{
"location": "/#ltrim",
"text": "ltrim(str) - Removes the leading space characters from str . ltrim(trimStr, str) - Removes the leading string contains the characters from the trim string Arguments: str - a string expression trimStr - the trim string characters to trim, the default value is a single space Examples: > SELECT ltrim(' SparkSQL ');\n SparkSQL\n> SELECT ltrim('Sp', 'SSparkSQLS');\n arkSQLS Since: 1.5.0",
"title": "ltrim"
},
{
"location": "/#map",
"text": "map(key0, value0, key1, value1, ...) - Creates a map with the given key/value pairs. Examples: > SELECT map(1.0, '2', 3.0, '4');\n {1.0:\"2\",3.0:\"4\"}",
"title": "map"
},
{
"location": "/#map_keys",
"text": "map_keys(map) - Returns an unordered array containing the keys of the map. Examples: > SELECT map_keys(map(1, 'a', 2, 'b'));\n [1,2]",
"title": "map_keys"
},
{
"location": "/#map_values",
"text": "map_values(map) - Returns an unordered array containing the values of the map. Examples: > SELECT map_values(map(1, 'a', 2, 'b'));\n [\"a\",\"b\"]",
"title": "map_values"
},
{
"location": "/#max",
"text": "max(expr) - Returns the maximum value of expr .",
"title": "max"
},
{
"location": "/#md5",
"text": "md5(expr) - Returns an MD5 128-bit checksum as a hex string of expr . Examples: > SELECT md5('Spark');\n 8cde774d6f7333752ed72cacddb05126",
"title": "md5"
},
{
"location": "/#mean",
"text": "mean(expr) - Returns the mean calculated from values of a group.",
"title": "mean"
},
{
"location": "/#min",
"text": "min(expr) - Returns the minimum value of expr .",
"title": "min"
},
{
"location": "/#minute",
"text": "minute(timestamp) - Returns the minute component of the string/timestamp. Examples: > SELECT minute('2009-07-30 12:58:59');\n 58 Since: 1.5.0",
"title": "minute"
},
{
"location": "/#mod",
"text": "expr1 mod expr2 - Returns the remainder after expr1 / expr2 . Examples: > SELECT 2 mod 1.8;\n 0.2\n> SELECT MOD(2, 1.8);\n 0.2",
"title": "mod"
},
{
"location": "/#monotonically_increasing_id",
"text": "monotonically_increasing_id() - Returns monotonically increasing 64-bit integers. The generated ID is guaranteed\nto be monotonically increasing and unique, but not consecutive. The current implementation\nputs the partition ID in the upper 31 bits, and the lower 33 bits represent the record number\nwithin each partition. The assumption is that the data frame has less than 1 billion\npartitions, and each partition has less than 8 billion records.",
"title": "monotonically_increasing_id"
},
{
"location": "/#month",
"text": "month(date) - Returns the month component of the date/timestamp. Examples: > SELECT month('2016-07-30');\n 7 Since: 1.5.0",
"title": "month"
},
{
"location": "/#months_between",
"text": "months_between(timestamp1, timestamp2) - Returns number of months between timestamp1 and timestamp2 . Examples: > SELECT months_between('1997-02-28 10:30:00', '1996-10-30');\n 3.94959677 Since: 1.5.0",
"title": "months_between"
},
{
"location": "/#named_struct",
"text": "named_struct(name1, val1, name2, val2, ...) - Creates a struct with the given field names and values. Examples: > SELECT named_struct(\"a\", 1, \"b\", 2, \"c\", 3);\n {\"a\":1,\"b\":2,\"c\":3}",
"title": "named_struct"
},
{
"location": "/#nanvl",
"text": "nanvl(expr1, expr2) - Returns expr1 if it's not NaN, or expr2 otherwise. Examples: > SELECT nanvl(cast('NaN' as double), 123);\n 123.0 Since: 1.5.0",
"title": "nanvl"
},
{
"location": "/#negative",
"text": "negative(expr) - Returns the negated value of expr . Examples: > SELECT negative(1);\n -1",
"title": "negative"
},
{
"location": "/#next_day",
"text": "next_day(start_date, day_of_week) - Returns the first date which is later than start_date and named as indicated. Examples: > SELECT next_day('2015-01-14', 'TU');\n 2015-01-20 Since: 1.5.0",
"title": "next_day"
},
{
"location": "/#not",
"text": "not expr - Logical not.",
"title": "not"
},
{
"location": "/#now",
"text": "now() - Returns the current timestamp at the start of query evaluation. Since: 1.5.0",
"title": "now"
},
{
"location": "/#ntile",
"text": "ntile(n) - Divides the rows for each window partition into n buckets ranging\nfrom 1 to at most n .",
"title": "ntile"
},
{
"location": "/#nullif",
"text": "nullif(expr1, expr2) - Returns null if expr1 equals to expr2 , or expr1 otherwise. Examples: > SELECT nullif(2, 2);\n NULL Since: 2.0.0",
"title": "nullif"
},
{
"location": "/#nvl",
"text": "nvl(expr1, expr2) - Returns expr2 if expr1 is null, or expr1 otherwise. Examples: > SELECT nvl(NULL, array('2'));\n [\"2\"] Since: 2.0.0",
"title": "nvl"
},
{
"location": "/#nvl2",
"text": "nvl2(expr1, expr2, expr3) - Returns expr2 if expr1 is not null, or expr3 otherwise. Examples: > SELECT nvl2(NULL, 2, 1);\n 1 Since: 2.0.0",
"title": "nvl2"
},
{
"location": "/#octet_length",
"text": "octet_length(expr) - Returns the byte length of string data or number of bytes of binary data. Examples: > SELECT octet_length('Spark SQL');\n 9 Since: 2.3.0",
"title": "octet_length"
},
{
"location": "/#or",
"text": "expr1 or expr2 - Logical OR.",
"title": "or"
},
{
"location": "/#parse_url",
"text": "parse_url(url, partToExtract[, key]) - Extracts a part from a URL. Examples: > SELECT parse_url('
http://spark.apache.org/path?query=1', 'HOST')\n spark.apache.org\n> SELECT parse_url('
http://spark.apache.org/path?query=1', 'QUERY')\n query=1\n> SELECT parse_url('
http://spark.apache.org/path?query=1', 'QUERY', 'query')\n 1 Since: 2.0.0",
"title": "parse_url"
},
{
"location": "/#percent_rank",
"text": "percent_rank() - Computes the percentage ranking of a value in a group of values.",
"title": "percent_rank"
},
{
"location": "/#percentile",
"text": "percentile(col, percentage [, frequency]) - Returns the exact percentile value of numeric column col at the given percentage. The value of percentage must be between 0.0 and 1.0. The\nvalue of frequency should be positive integral percentile(col, array(percentage1 [, percentage2]...) [, frequency]) - Returns the exact\npercentile value array of numeric column col at the given percentage(s). Each value\nof the percentage array must be between 0.0 and 1.0. The value of frequency should be\npositive integral",
"title": "percentile"
},
{
"location": "/#percentile_approx",
"text": "percentile_approx(col, percentage [, accuracy]) - Returns the approximate percentile value of numeric\ncolumn col at the given percentage. The value of percentage must be between 0.0\nand 1.0. The accuracy parameter (default: 10000) is a positive numeric literal which\ncontrols approximation accuracy at the cost of memory. Higher value of accuracy yields\nbetter accuracy, 1.0/accuracy is the relative error of the approximation.\nWhen percentage is an array, each value of the percentage array must be between 0.0 and 1.0.\nIn this case, returns the approximate percentile array of column col at the given\npercentage array. Examples: > SELECT percentile_approx(10.0, array(0.5, 0.4, 0.1), 100);\n [10.0,10.0,10.0]\n> SELECT percentile_approx(10.0, 0.5, 100);\n 10.0",
"title": "percentile_approx"
},
{
"location": "/#pi",
"text": "pi() - Returns pi. Examples: > SELECT pi();\n 3.141592653589793",
"title": "pi"
},
{
"location": "/#pmod",
"text": "pmod(expr1, expr2) - Returns the positive value of expr1 mod expr2 . Examples: > SELECT pmod(10, 3);\n 1\n> SELECT pmod(-10, 3);\n 2",
"title": "pmod"
},
{
"location": "/#posexplode",
"text": "posexplode(expr) - Separates the elements of array expr into multiple rows with positions, or the elements of map expr into multiple rows and columns with positions. Examples: > SELECT posexplode(array(10,20));\n 0 10\n 1 20",
"title": "posexplode"
},
{
"location": "/#posexplode_outer",
"text": "posexplode_outer(expr) - Separates the elements of array expr into multiple rows with positions, or the elements of map expr into multiple rows and columns with positions. Examples: > SELECT posexplode_outer(array(10,20));\n 0 10\n 1 20",
"title": "posexplode_outer"
},
{
"location": "/#position",
"text": "position(substr, str[, pos]) - Returns the position of the first occurrence of substr in str after position pos .\nThe given pos and return value are 1-based. Examples: > SELECT position('bar', 'foobarbar');\n 4\n> SELECT position('bar', 'foobarbar', 5);\n 7\n> SELECT POSITION('bar' IN 'foobarbar');\n 4 Since: 1.5.0",
"title": "position"
},
{
"location": "/#positive",
"text": "positive(expr) - Returns the value of expr .",
"title": "positive"
},
{
"location": "/#pow",
"text": "pow(expr1, expr2) - Raises expr1 to the power of expr2 . Examples: > SELECT pow(2, 3);\n 8.0",
"title": "pow"
},
{
"location": "/#power",
"text": "power(expr1, expr2) - Raises expr1 to the power of expr2 . Examples: > SELECT power(2, 3);\n 8.0",
"title": "power"
},
{
"location": "/#printf",
"text": "printf(strfmt, obj, ...) - Returns a formatted string from printf-style format strings. Examples: > SELECT printf(\"Hello World %d %s\", 100, \"days\");\n Hello World 100 days Since: 1.5.0",
"title": "printf"
},
{
"location": "/#quarter",
"text": "quarter(date) - Returns the quarter of the year for date, in the range 1 to 4. Examples: > SELECT quarter('2016-08-31');\n 3 Since: 1.5.0",
"title": "quarter"
},
{
"location": "/#radians",
"text": "radians(expr) - Converts degrees to radians. Arguments: expr - angle in degrees Examples: > SELECT radians(180);\n 3.141592653589793",
"title": "radians"
},
{
"location": "/#rand",
"text": "rand([seed]) - Returns a random value with independent and identically distributed (i.i.d.) uniformly distributed values in [0, 1). Examples: > SELECT rand();\n 0.9629742951434543\n> SELECT rand(0);\n 0.8446490682263027\n> SELECT rand(null);\n 0.8446490682263027 Since: 1.5.0",
"title": "rand"
},
{
"location": "/#randn",
"text": "randn([seed]) - Returns a random value with independent and identically distributed (i.i.d.) values drawn from the standard normal distribution. Examples: > SELECT randn();\n -0.3254147983080288\n> SELECT randn(0);\n 1.1164209726833079\n> SELECT randn(null);\n 1.1164209726833079 Since: 1.5.0",
"title": "randn"
},
{
"location": "/#rank",
"text": "rank() - Computes the rank of a value in a group of values. The result is one plus the number\nof rows preceding or equal to the current row in the ordering of the partition. The values\nwill produce gaps in the sequence.",
"title": "rank"
},
{
"location": "/#reflect",
"text": "reflect(class, method[, arg1[, arg2 ..]]) - Calls a method with reflection. Examples: > SELECT reflect('java.util.UUID', 'randomUUID');\n c33fb387-8500-4bfa-81d2-6e0e3e930df2\n> SELECT reflect('java.util.UUID', 'fromString', 'a5cf6c42-0c85-418f-af6c-3e4e5b1328f2');\n a5cf6c42-0c85-418f-af6c-3e4e5b1328f2",
"title": "reflect"
},
{
"location": "/#regexp_extract",
"text": "regexp_extract(str, regexp[, idx]) - Extracts a group that matches regexp . Examples: > SELECT regexp_extract('100-200', '(\\d+)-(\\d+)', 1);\n 100 Since: 1.5.0",
"title": "regexp_extract"
},
{
"location": "/#regexp_replace",
"text": "regexp_replace(str, regexp, rep) - Replaces all substrings of str that match regexp with rep . Examples: > SELECT regexp_replace('100-200', '(\\d+)', 'num');\n num-num Since: 1.5.0",
"title": "regexp_replace"
},
{
"location": "/#repeat",
"text": "repeat(str, n) - Returns the string which repeats the given string value n times. Examples: > SELECT repeat('123', 2);\n 123123 Since: 1.5.0",
"title": "repeat"
},
{
"location": "/#replace",
"text": "replace(str, search[, replace]) - Replaces all occurrences of search with replace . Arguments: str - a string expression search - a string expression. If search is not found in str , str is returned unchanged. replace - a string expression. If replace is not specified or is an empty string, nothing replaces\n the string that is removed from str . Examples: > SELECT replace('ABCabc', 'abc', 'DEF');\n ABCDEF Since: 2.3.0",
"title": "replace"
},
{
"location": "/#reverse",
"text": "reverse(str) - Returns the reversed given string. Examples: > SELECT reverse('Spark SQL');\n LQS krapS",
"title": "reverse"
},
{
"location": "/#right",
"text": "right(str, len) - Returns the rightmost len ( len can be string type) characters from the string str ,if len is less or equal than 0 the result is an empty string. Examples: > SELECT right('Spark SQL', 3);\n SQL Since: 2.3.0",
"title": "right"
},
{
"location": "/#rint",
"text": "rint(expr) - Returns the double value that is closest in value to the argument and is equal to a mathematical integer. Examples: > SELECT rint(12.3456);\n 12.0",
"title": "rint"
},
{
"location": "/#rlike",
"text": "str rlike regexp - Returns true if str matches regexp , or false otherwise. Arguments: str - a string expression regexp - a string expression. The pattern string should be a Java regular expression. Since Spark 2.0, string literals (including regex patterns) are unescaped in our SQL\nparser. For example, to match \"\\abc\", a regular expression for regexp can be\n\"^\\abc$\". There is a SQL config 'spark.sql.parser.escapedStringLiterals' that can be used to\nfallback to the Spark 1.6 behavior regarding string literal parsing. For example,\nif the config is enabled, the regexp that can match \"\\abc\" is \"^\\abc$\". Examples: When spark.sql.parser.escapedStringLiterals is disabled (default).\n> SELECT '%SystemDrive%\\Users\\John' rlike '%SystemDrive%\\\\Users.*'\ntrue\n\nWhen spark.sql.parser.escapedStringLiterals is enabled.\n> SELECT '%SystemDrive%\\Users\\John' rlike '%SystemDrive%\\Users.*'\ntrue Note: Use LIKE to match with simple string pattern. Since: 1.0.0",
"title": "rlike"
},
{
"location": "/#rollup",
"text": "rollup([col1[, col2 ..]]) - create a multi-dimensional rollup using the specified columns\nso that we can run aggregation on them. Examples: > SELECT name, age, count(*) FROM VALUES (2, 'Alice'), (5, 'Bob') people(age, name) GROUP BY rollup(name, age);\n NULL NULL 2\n Alice 2 1\n Bob 5 1\n Bob NULL 1\n Alice NULL 1 Since: 2.0.0",
"title": "rollup"
},
{
"location": "/#round",
"text": "round(expr, d) - Returns expr rounded to d decimal places using HALF_UP rounding mode. Examples: > SELECT round(2.5, 0);\n 3.0",
"title": "round"
},
{
"location": "/#row_number",
"text": "row_number() - Assigns a unique, sequential number to each row, starting with one,\naccording to the ordering of rows within the window partition.",
"title": "row_number"
},
{
"location": "/#rpad",
"text": "rpad(str, len, pad) - Returns str , right-padded with pad to a length of len .\nIf str is longer than len , the return value is shortened to len characters. Examples: > SELECT rpad('hi', 5, '??');\n hi???\n> SELECT rpad('hi', 1, '??');\n h Since: 1.5.0",
"title": "rpad"
},
{
"location": "/#rtrim",
"text": "rtrim(str) - Removes the trailing space characters from str . rtrim(trimStr, str) - Removes the trailing string which contains the characters from the trim string from the str Arguments: str - a string expression trimStr - the trim string characters to trim, the default value is a single space Examples: > SELECT rtrim(' SparkSQL ');\n SparkSQL\n> SELECT rtrim('LQSa', 'SSparkSQLS');\n SSpark Since: 1.5.0",
"title": "rtrim"
},
{
"location": "/#second",
"text": "second(timestamp) - Returns the second component of the string/timestamp. Examples: > SELECT second('2009-07-30 12:58:59');\n 59 Since: 1.5.0",
"title": "second"
},
{
"location": "/#sentences",
"text": "sentences(str[, lang, country]) - Splits str into an array of array of words. Examples: > SELECT sentences('Hi there! Good morning.');\n [[\"Hi\",\"there\"],[\"Good\",\"morning\"]] Since: 2.0.0",
"title": "sentences"
},
{
"location": "/#sha",
"text": "sha(expr) - Returns a sha1 hash value as a hex string of the expr . Examples: > SELECT sha('Spark');\n 85f5955f4b27a9a4c2aab6ffe5d7189fc298b92c",
"title": "sha"
},
{
"location": "/#sha1",
"text": "sha1(expr) - Returns a sha1 hash value as a hex string of the expr . Examples: > SELECT sha1('Spark');\n 85f5955f4b27a9a4c2aab6ffe5d7189fc298b92c",
"title": "sha1"
},
{
"location": "/#sha2",
"text": "sha2(expr, bitLength) - Returns a checksum of SHA-2 family as a hex string of expr .\nSHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256. Examples: > SELECT sha2('Spark', 256);\n 529bc3b07127ecb7e53a4dcf1991d9152c24537d919178022b2c42657f79a26b",
"title": "sha2"
},
{
"location": "/#shiftleft",
"text": "shiftleft(base, expr) - Bitwise left shift. Examples: > SELECT shiftleft(2, 1);\n 4",
"title": "shiftleft"
},
{
"location": "/#shiftright",
"text": "shiftright(base, expr) - Bitwise (signed) right shift. Examples: > SELECT shiftright(4, 1);\n 2",
"title": "shiftright"
},
{
"location": "/#shiftrightunsigned",
"text": "shiftrightunsigned(base, expr) - Bitwise unsigned right shift. Examples: > SELECT shiftrightunsigned(4, 1);\n 2",
"title": "shiftrightunsigned"
},
{
"location": "/#sign",
"text": "sign(expr) - Returns -1.0, 0.0 or 1.0 as expr is negative, 0 or positive. Examples: > SELECT sign(40);\n 1.0",
"title": "sign"
},
{
"location": "/#signum",
"text": "signum(expr) - Returns -1.0, 0.0 or 1.0 as expr is negative, 0 or positive. Examples: > SELECT signum(40);\n 1.0",
"title": "signum"
},
{
"location": "/#sin",
"text": "sin(expr) - Returns the sine of expr , as if computed by java.lang.Math.sin . Arguments: expr - angle in radians Examples: > SELECT sin(0);\n 0.0",
"title": "sin"
},
{
"location": "/#sinh",
"text": "sinh(expr) - Returns hyperbolic sine of expr , as if computed by java.lang.Math.sinh . Arguments: expr - hyperbolic angle Examples: > SELECT sinh(0);\n 0.0",
"title": "sinh"
},
{
"location": "/#size",
"text": "size(expr) - Returns the size of an array or a map. Returns -1 if null. Examples: > SELECT size(array('b', 'd', 'c', 'a'));\n 4",
"title": "size"
},
{
"location": "/#skewness",
"text": "skewness(expr) - Returns the skewness value calculated from values of a group.",
"title": "skewness"
},
{
"location": "/#smallint",
"text": "smallint(expr) - Casts the value expr to the target data type smallint .",
"title": "smallint"
},
{
"location": "/#sort_array",
"text": "sort_array(array[, ascendingOrder]) - Sorts the input array in ascending or descending order according to the natural ordering of the array elements. Examples: > SELECT sort_array(array('b', 'd', 'c', 'a'), true);\n [\"a\",\"b\",\"c\",\"d\"]",
"title": "sort_array"
},
{
"location": "/#soundex",
"text": "soundex(str) - Returns Soundex code of the string. Examples: > SELECT soundex('Miller');\n M460 Since: 1.5.0",
"title": "soundex"
},
{
"location": "/#space",
"text": "space(n) - Returns a string consisting of n spaces. Examples: > SELECT concat(space(2), '1');\n 1 Since: 1.5.0",
"title": "space"
},
{
"location": "/#spark_partition_id",
"text": "spark_partition_id() - Returns the current partition id.",
"title": "spark_partition_id"
},
{
"location": "/#split",
"text": "split(str, regex) - Splits str around occurrences that match regex . Examples: > SELECT split('oneAtwoBthreeC', '[ABC]');\n [\"one\",\"two\",\"three\",\"\"] Since: 1.5.0",
"title": "split"
},
{
"location": "/#sqrt",
"text": "sqrt(expr) - Returns the square root of expr . Examples: > SELECT sqrt(4);\n 2.0",
"title": "sqrt"
},
{
"location": "/#stack",
"text": "stack(n, expr1, ..., exprk) - Separates expr1 , ..., exprk into n rows. Examples: > SELECT stack(2, 1, 2, 3);\n 1 2\n 3 NULL",
"title": "stack"
},
{
"location": "/#std",
"text": "std(expr) - Returns the sample standard deviation calculated from values of a group.",
"title": "std"
},
{
"location": "/#stddev",
"text": "stddev(expr) - Returns the sample standard deviation calculated from values of a group.",
"title": "stddev"
},
{
"location": "/#stddev_pop",
"text": "stddev_pop(expr) - Returns the population standard deviation calculated from values of a group.",
"title": "stddev_pop"
},
{
"location": "/#stddev_samp",
"text": "stddev_samp(expr) - Returns the sample standard deviation calculated from values of a group.",
"title": "stddev_samp"
},
{
"location": "/#str_to_map",
"text": "str_to_map(text[, pairDelim[, keyValueDelim]]) - Creates a map after splitting the text into key/value pairs using delimiters. Default delimiters are ',' for pairDelim and ':' for keyValueDelim . Examples: > SELECT str_to_map('a:1,b:2,c:3', ',', ':');\n map(\"a\":\"1\",\"b\":\"2\",\"c\":\"3\")\n> SELECT str_to_map('a');\n map(\"a\":null)",
"title": "str_to_map"
},
{
"location": "/#string",
"text": "string(expr) - Casts the value expr to the target data type string .",
"title": "string"
},
{
"location": "/#struct",
"text": "struct(col1, col2, col3, ...) - Creates a struct with the given field values.",
"title": "struct"
},
{
"location": "/#substr",
"text": "substr(str, pos[, len]) - Returns the substring of str that starts at pos and is of length len , or the slice of byte array that starts at pos and is of length len . Examples: > SELECT substr('Spark SQL', 5);\n k SQL\n> SELECT substr('Spark SQL', -3);\n SQL\n> SELECT substr('Spark SQL', 5, 1);\n k Since: 1.5.0",
"title": "substr"
},
{
"location": "/#substring",
"text": "substring(str, pos[, len]) - Returns the substring of str that starts at pos and is of length len , or the slice of byte array that starts at pos and is of length len . Examples: > SELECT substring('Spark SQL', 5);\n k SQL\n> SELECT substring('Spark SQL', -3);\n SQL\n> SELECT substring('Spark SQL', 5, 1);\n k Since: 1.5.0",
"title": "substring"
},
{
"location": "/#substring_index",
"text": "substring_index(str, delim, count) - Returns the substring from str before count occurrences of the delimiter delim .\nIf count is positive, everything to the left of the final delimiter (counting from the\nleft) is returned. If count is negative, everything to the right of the final delimiter\n(counting from the right) is returned. The function substring_index performs a case-sensitive match\nwhen searching for delim . Examples: > SELECT substring_index('www.apache.org', '.', 2);\n www.apache Since: 1.5.0",
"title": "substring_index"
},
{
"location": "/#sum",
"text": "sum(expr) - Returns the sum calculated from values of a group.",
"title": "sum"
},
{
"location": "/#tan",
"text": "tan(expr) - Returns the tangent of expr , as if computed by java.lang.Math.tan . Arguments: expr - angle in radians Examples: > SELECT tan(0);\n 0.0",
"title": "tan"
},
{
"location": "/#tanh",
"text": "tanh(expr) - Returns the hyperbolic tangent of expr , as if computed by java.lang.Math.tanh . Arguments: expr - hyperbolic angle Examples: > SELECT tanh(0);\n 0.0",
"title": "tanh"
},
{
"location": "/#timestamp",
"text": "timestamp(expr) - Casts the value expr to the target data type timestamp .",
"title": "timestamp"
},
{
"location": "/#tinyint",
"text": "tinyint(expr) - Casts the value expr to the target data type tinyint .",
"title": "tinyint"
},
{
"location": "/#to_date",
"text": "to_date(date_str[, fmt]) - Parses the date_str expression with the fmt expression to\na date. Returns null with invalid input. By default, it follows casting rules to a date if\nthe fmt is omitted. Examples: > SELECT to_date('2009-07-30 04:17:52');\n 2009-07-30\n> SELECT to_date('2016-12-31', 'yyyy-MM-dd');\n 2016-12-31 Since: 1.5.0",
"title": "to_date"
},
{
"location": "/#to_json",
"text": "to_json(expr[, options]) - Returns a json string with a given struct value Examples: > SELECT to_json(named_struct('a', 1, 'b', 2));\n {\"a\":1,\"b\":2}\n> SELECT to_json(named_struct('time', to_timestamp('2015-08-26', 'yyyy-MM-dd')), map('timestampFormat', 'dd/MM/yyyy'));\n {\"time\":\"26/08/2015\"}\n> SELECT to_json(array(named_struct('a', 1, 'b', 2)));\n [{\"a\":1,\"b\":2}]\n> SELECT to_json(map('a', named_struct('b', 1)));\n {\"a\":{\"b\":1}}\n> SELECT to_json(map(named_struct('a', 1),named_struct('b', 2)));\n {\"[1]\":{\"b\":2}}\n> SELECT to_json(map('a', 1));\n {\"a\":1}\n> SELECT to_json(array((map('a', 1))));\n [{\"a\":1}] Since: 2.2.0",
"title": "to_json"
},
{
"location": "/#to_timestamp",
"text": "to_timestamp(timestamp[, fmt]) - Parses the timestamp expression with the fmt expression to\na timestamp. Returns null with invalid input. By default, it follows casting rules to\na timestamp if the fmt is omitted. Examples: > SELECT to_timestamp('2016-12-31 00:12:00');\n 2016-12-31 00:12:00\n> SELECT to_timestamp('2016-12-31', 'yyyy-MM-dd');\n 2016-12-31 00:00:00 Since: 2.2.0",
"title": "to_timestamp"
},
{
"location": "/#to_unix_timestamp",
"text": "to_unix_timestamp(expr[, pattern]) - Returns the UNIX timestamp of the given time. Examples: > SELECT to_unix_timestamp('2016-04-08', 'yyyy-MM-dd');\n 1460041200 Since: 1.6.0",
"title": "to_unix_timestamp"
},
{
"location": "/#to_utc_timestamp",
"text": "to_utc_timestamp(timestamp, timezone) - Given a timestamp like '2017-07-14 02:40:00.0', interprets it as a time in the given time zone, and renders that time as a timestamp in UTC. For example, 'GMT+1' would yield '2017-07-14 01:40:00.0'. Examples: > SELECT to_utc_timestamp('2016-08-31', 'Asia/Seoul');\n 2016-08-30 15:00:00 Since: 1.5.0",
"title": "to_utc_timestamp"
},
{
"location": "/#translate",
"text": "translate(input, from, to) - Translates the input string by replacing the characters present in the from string with the corresponding characters in the to string. Examples: > SELECT translate('AaBbCc', 'abc', '123');\n A1B2C3 Since: 1.5.0",
"title": "translate"
},
{
"location": "/#trim",
"text": "trim(str) - Removes the leading and trailing space characters from str . trim(BOTH trimStr FROM str) - Remove the leading and trailing trimStr characters from str trim(LEADING trimStr FROM str) - Remove the leading trimStr characters from str trim(TRAILING trimStr FROM str) - Remove the trailing trimStr characters from str Arguments: str - a string expression trimStr - the trim string characters to trim, the default value is a single space BOTH, FROM - these are keywords to specify trimming string characters from both ends of\n the string LEADING, FROM - these are keywords to specify trimming string characters from the left\n end of the string TRAILING, FROM - these are keywords to specify trimming string characters from the right\n end of the string Examples: > SELECT trim(' SparkSQL ');\n SparkSQL\n> SELECT trim('SL', 'SSparkSQLS');\n parkSQ\n> SELECT trim(BOTH 'SL' FROM 'SSparkSQLS');\n parkSQ\n> SELECT trim(LEADING 'SL' FROM 'SSparkSQLS');\n parkSQLS\n> SELECT trim(TRAILING 'SL' FROM 'SSparkSQLS');\n SSparkSQ Since: 1.5.0",
"title": "trim"
},
{
"location": "/#trunc",
"text": "trunc(date, fmt) - Returns date with the time portion of the day truncated to the unit specified by the format model fmt . fmt should be one of [\"year\", \"yyyy\", \"yy\", \"mon\", \"month\", \"mm\"] Examples: > SELECT trunc('2009-02-12', 'MM');\n 2009-02-01\n> SELECT trunc('2015-10-27', 'YEAR');\n 2015-01-01 Since: 1.5.0",
"title": "trunc"
},
{
"location": "/#ucase",
"text": "ucase(str) - Returns str with all characters changed to uppercase. Examples: > SELECT ucase('SparkSql');\n SPARKSQL Since: 1.0.1",
"title": "ucase"
},
{
"location": "/#unbase64",
"text": "unbase64(str) - Converts the argument from a base 64 string str to a binary. Examples: > SELECT unbase64('U3BhcmsgU1FM');\n Spark SQL Since: 1.5.0",
"title": "unbase64"
},
{
"location": "/#unhex",
"text": "unhex(expr) - Converts hexadecimal expr to binary. Examples: > SELECT decode(unhex('537061726B2053514C'), 'UTF-8');\n Spark SQL",
"title": "unhex"
},
{
"location": "/#unix_timestamp",
"text": "unix_timestamp([expr[, pattern]]) - Returns the UNIX timestamp of current or specified time. Examples: > SELECT unix_timestamp();\n 1476884637\n> SELECT unix_timestamp('2016-04-08', 'yyyy-MM-dd');\n 1460041200 Since: 1.5.0",
"title": "unix_timestamp"
},
{
"location": "/#upper",
"text": "upper(str) - Returns str with all characters changed to uppercase. Examples: > SELECT upper('SparkSql');\n SPARKSQL Since: 1.0.1",
"title": "upper"
},
{
"location": "/#uuid",
"text": "uuid() - Returns an universally unique identifier (UUID) string. The value is returned as a canonical UUID 36-character string. Examples: > SELECT uuid();\n 46707d92-02f4-4817-8116-a4c3b23e6266",
"title": "uuid"
},
{
"location": "/#var_pop",
"text": "var_pop(expr) - Returns the population variance calculated from values of a group.",
"title": "var_pop"
},
{
"location": "/#var_samp",
"text": "var_samp(expr) - Returns the sample variance calculated from values of a group.",
"title": "var_samp"
},
{
"location": "/#variance",
"text": "variance(expr) - Returns the sample variance calculated from values of a group.",
"title": "variance"
},
{
"location": "/#weekofyear",
"text": "weekofyear(date) - Returns the week of the year of the given date. A week is considered to start on a Monday and week 1 is the first week with >3 days. Examples: > SELECT weekofyear('2008-02-20');\n 8 Since: 1.5.0",
"title": "weekofyear"
},
{
"location": "/#when",
"text": "CASE WHEN expr1 THEN expr2 [WHEN expr3 THEN expr4]* [ELSE expr5] END - When expr1 = true, returns expr2 ; else when expr3 = true, returns expr4 ; else returns expr5 . Arguments: expr1, expr3 - the branch condition expressions should all be boolean type. expr2, expr4, expr5 - the branch value expressions and else value expression should all be\n same type or coercible to a common type. Examples: > SELECT CASE WHEN 1 > 0 THEN 1 WHEN 2 > 0 THEN 2.0 ELSE 1.2 END;\n 1\n> SELECT CASE WHEN 1 < 0 THEN 1 WHEN 2 > 0 THEN 2.0 ELSE 1.2 END;\n 2\n> SELECT CASE WHEN 1 < 0 THEN 1 WHEN 2 < 0 THEN 2.0 END;\n NULL",
"title": "when"
},
{
"location": "/#window",
"text": "",
"title": "window"
},
{
"location": "/#xpath",
"text": "xpath(xml, xpath) - Returns a string array of values within the nodes of xml that match the XPath expression. Examples: > SELECT xpath('<a><b>b1</b><b>b2</b><b>b3</b><c>c1</c><c>c2</c></a>','a/b/text()');\n ['b1','b2','b3']",
"title": "xpath"
},
{
"location": "/#xpath_boolean",
"text": "xpath_boolean(xml, xpath) - Returns true if the XPath expression evaluates to true, or if a matching node is found. Examples: > SELECT xpath_boolean('<a><b>1</b></a>','a/b');\n true",
"title": "xpath_boolean"
},
{
"location": "/#xpath_double",
"text": "xpath_double(xml, xpath) - Returns a double value, the value zero if no match is found, or NaN if a match is found but the value is non-numeric. Examples: > SELECT xpath_double('<a><b>1</b><b>2</b></a>', 'sum(a/b)');\n 3.0",
"title": "xpath_double"
},
{
"location": "/#xpath_float",
"text": "xpath_float(xml, xpath) - Returns a float value, the value zero if no match is found, or NaN if a match is found but the value is non-numeric. Examples: > SELECT xpath_float('<a><b>1</b><b>2</b></a>', 'sum(a/b)');\n 3.0",
"title": "xpath_float"
},
{
"location": "/#xpath_int",
"text": "xpath_int(xml, xpath) - Returns an integer value, or the value zero if no match is found, or a match is found but the value is non-numeric. Examples: > SELECT xpath_int('<a><b>1</b><b>2</b></a>', 'sum(a/b)');\n 3",
"title": "xpath_int"
},
{
"location": "/#xpath_long",
"text": "xpath_long(xml, xpath) - Returns a long integer value, or the value zero if no match is found, or a match is found but the value is non-numeric. Examples: > SELECT xpath_long('<a><b>1</b><b>2</b></a>', 'sum(a/b)');\n 3",
"title": "xpath_long"
},
{
"location": "/#xpath_number",
"text": "xpath_number(xml, xpath) - Returns a double value, the value zero if no match is found, or NaN if a match is found but the value is non-numeric. Examples: > SELECT xpath_number('<a><b>1</b><b>2</b></a>', 'sum(a/b)');\n 3.0",
"title": "xpath_number"
},
{
"location": "/#xpath_short",
"text": "xpath_short(xml, xpath) - Returns a short integer value, or the value zero if no match is found, or a match is found but the value is non-numeric. Examples: > SELECT xpath_short('<a><b>1</b><b>2</b></a>', 'sum(a/b)');\n 3",
"title": "xpath_short"
},
{
"location": "/#xpath_string",
"text": "xpath_string(xml, xpath) - Returns the text contents of the first xml node that matches the XPath expression. Examples: > SELECT xpath_string('<a><b>b</b><c>cc</c></a>','a/c');\n cc",
"title": "xpath_string"
},
{
"location": "/#year",
"text": "year(date) - Returns the year component of the date/timestamp. Examples: > SELECT year('2016-07-30');\n 2016 Since: 1.5.0",
"title": "year"
},
{
"location": "/#_15",
"text": "expr1 | expr2 - Returns the result of bitwise OR of expr1 and expr2 . Examples: > SELECT 3 | 5;\n 7",
"title": "|"
},
{
"location": "/#_16",
"text": "~ expr - Returns the result of bitwise NOT of expr . Examples: > SELECT ~ 0;\n -1",
"title": "~"
}
]
}
