Tutorials
Courses

Similar Topics

Web Technologies37.4K+ articles

DSA21.3K+ articles

Python21.3K+ articles

Experiences16.6K+ articles

Interview Experiences14.2K+ articles

Java10.0K+ articles

Scala1.8K+ articles

Scala1.2K+ articles

Python-Pyspark179+ articles

Hadoop41+ articles

Apache-spark

11 posts

Install Apache Spark in a Standalone Mode on Windows

Last Updated: 10 April 2023

Apache Spark is a lightning-fast unified analytics engine used for cluster computing for large data sets like BigData and Hadoop with the aim to run programs parallel acro...read more

article_image

How to Create Multiple Lags in PySpark

Last Updated: 28 April 2025

In this article, we are going to learn how to create multiple lags using pyspark in Python.What is lag in Pyspark? The lag lets our query on more than one row of a table a...read more

article_image

Technical Scripter

Python Programs

Technical Scripter 2022

Python Framework

How to add a column to a nested struct in a pyspark

Last Updated: 28 April 2025

In this article, we are going to learn how to add a column to a nested struct using Pyspark in Python.Have you ever worked in a Pyspark data frame? If yes, then might sure...read more

article_image

Technical Scripter

Python Programs

Technical Scripter 2022

Python Framework

Drop a column with same name using column index in PySpark

Last Updated: 28 April 2025

In this article, we are going to learn how to drop a column with the same name using column index using Pyspark in Python.Pyspark offers you the essential function 'drop' ...read more

article_image

Technical Scripter

Python Programs

Technical Scripter 2022

Python Framework

PySpark lag() Function

Last Updated: 28 April 2025

The function that allows the user to query on more than one row of a table returning the previous row in the table is known as lag in Python. Apart from returning the offs...read more

article_image

Technical Scripter

Python Programs

Technical Scripter 2022

Spark - Split array to separate column

Last Updated: 28 April 2025

Apache Spark is a potent big data processing system that can analyze enormous amounts of data concurrently over distributed computer clusters. PySpark is a Python-based in...read more

article_image

Identify corrupted records in a dataset using pyspark

Last Updated: 07 April 2025

There can be datasets that may contain corrupt records. Those records don't follow data-specific rules that are followed by correct records e.g., a corrupt record may have...read more

article_image

Spark vs Impala

Last Updated: 01 December 2023

Spark and Impala are the two most common tools used for big data analytics. This article focuses on discussing the pros, cons, and differences between the two tools.What i...read more

Geeks Premier League

Software Testing

Geeks Premier League 2023

Top 7 Big Data Frameworks in 2025

Last Updated: 13 November 2024

Big data, or the enormous amount of information that is always expanding, presents both challenges and opportunities for businesses. Robust big-data frameworks are necessa...read more

article_image

Apache Flink vs Apache Spark: Top Differences

Last Updated: 24 April 2025

Apache Flink and Apache Spark are two well-liked competitors in the rapidly growing field of big data, where information flows like a roaring torrent. These distributed pr...read more

article_image

Wide and Narrow Dependencies in Apache Spark

Last Updated: 03 May 2024

Apache Spark, a powerful distributed computing framework, is designed to process large-scale datasets efficiently across a cluster of machines. However, Dependencies play ...read more

article_image

Data Engineering