In Depth understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames, RDDs for Pyspark and PANDA libraries Hands on with strong development skills in the area of SQL, UNIX Shell scripting, Linux, Oracle, PL/SQL, Teradata, SQL Server, Perl and Python scripting.Experience in NoSQL column-oriented database and its integration with Hadoop Cluster. Experience data processing like collecting, aggregating, moving from various sources using Spark (PySpark) and Panda libraries. Migrating the coding from hive to Oracle using Python SPARK SQL (PySpark), Data Frames and RDD.Experience in creating tables, partitioning, bucketing, loading and aggregating data using Hive. Expertise in transferring data from RDBMS to HDFS and HIVE table using SQOOP and SPARK server.Expertise in using Spark-SQL with various data sources like Json and Hive.
Experience in using Accumulator variable, broadcast variable, RDD caching for Spark streaming.In-Depth understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames, Spark Streaming with PySpark and PANDA library.Around 5+ years of experience as HDFS/PySpark developer using Big Data Technologies like Hadoop Ecosystems, Spark Ecosystems.Possess subject matter expertise in Teradata FSDM/FSLDM, Complaisance and Trade Surveillance, Risk Data, Wholesale Customer data domain, customer complaints and surveillance.Strong industry leader with 17 years of experience covering Data Management diversified areas covering Data Architecture, Database Implementation - SQL Development, Data Modeling and design, Data Quality, Meta Data Management, Data Profiling and Data Governance along with extensive experience in implementing E nterprise Data Warehousing and Master Data for financial and other sectors.
0 kommentar(er)
