Job Description

Job Title: Hadoop Admin

Location: Denver, CO
 

Role: Hadoop Administrator

Industry Type: Information Technology

Department: Data Engineering

Employment Type: Full-time

Role Category: IT Infrastructure Management

Job Description: 

We are seeking an experienced Hadoop Admin to join our team in Denver, CO. The ideal candidate will have a strong background in managing and maintaining Hadoop clusters, ensuring their performance, reliability, and security. You will work closely with data engineers and analysts to support big data initiatives and optimize our data infrastructure.

Key Responsibilities:

• Install, configure, and maintain Hadoop clusters, including HDFS, YARN, MapReduce, and related components.
• Monitor cluster performance and troubleshoot issues to ensure high availability and reliability.
• Implement and manage data security policies and procedures.
• Perform regular system maintenance, including software updates and patches.
• Responsible for managing storage, optimizing performance, and handling volume management of Hadoop clusters and MapReduce operations.
• Tasks include designing, capacity planning, cluster setup, performance optimization, monitoring, architectural planning, scaling, and overall administration.
• Oversee connectivity and performance of the Hadoop cluster.
• Perform SQL and system-wide performance tuning.
• Create documentation for new environments and establish organizational standards to ensure a resilient infrastructure.
• Collaborate with data engineers to optimize data processing workflows.
• Manage and monitor data storage and backup solutions.
• Develop and maintain documentation for Hadoop cluster configurations and procedures.
• Stay updated with the latest advancements in Hadoop and big data technologies.
   

Qualifications:

• Bachelor’s degree in Computer Science, Information Technology, or a related field.
• Proven experience as a Hadoop Admin or similar role.
• Strong knowledge of Hadoop ecosystem components (HDFS, YARN, MapReduce, Hive, HBase, etc.).
• Proficiency in Linux/Unix system administration.
• Experience with scripting languages such as Python, Bash, or Perl.
• Excellent problem-solving and analytical skills.
• Strong communication and teamwork abilities.
   

Job Role:

• Strong understanding of SDLC and agile methodologies.
• Skilled in installing and configuring Hadoop clusters, including components like HDFS, MapReduce, Hive, Pig, HBase, and related tools.
• Experience in managing and monitoring Hadoop clusters to ensure optimal performance and high availability.
• Expertise in planning and executing data backup and disaster recovery strategies for Hadoop clusters.
• Proactively monitor and tune Hadoop cluster performance to optimize resource use and eliminate bottlenecks.
• Provide technical support to both developers and end-users when required.
• Stay informed about the latest technologies and industry trends.
• Strong problem-solving and logical thinking skills, with a proven ability to collaborate effectively.
   

Experience requirement

• Capable of deploying Hadoop clusters, managing node additions and removals, monitoring job execution, tracking critical cluster components, configuring name-node high availability, scheduling tasks, and performing backups.
• Proficient in installing and managing Hadoop distributions, such as Hortonworks and Cloudera.
• Experienced in utilizing and troubleshooting Open Source technologies.
• Proven ability to collaborate effectively with both application and infrastructure teams.
• Skilled in reviewing current database infrastructure and identifying opportunities for improvement.
• Able to implement software lifecycle methodologies to ensure adherence to supported releases and development roadmaps.
   

Education Qualification:

UG: Any Degree

PG: Any Degree
 

Key Skills:

1. Hadoop Ecosystem Proficiency

In-depth knowledge of Hadoop components such as HDFS, YARN, MapReduce, Hive, Pig, HBase, Oozie, and Zookeeper, with the ability to install, configure, and manage these tools.

2. Cluster Management

Expertise in setting up, configuring, and maintaining large-scale Hadoop clusters, ensuring optimal performance, high availability, and fault tolerance.

3. System Monitoring & Troubleshooting

Expertise in using monitoring tools like Ambari, Cloudera Manager, or Nagios to track cluster performance, detect anomalies, and troubleshoot issues.

4. Linux/Unix Administration

Strong skills in Linux/Unix environments, essential for managing the underlying systems supporting Hadoop clusters.

5. Networking & Storage Management

Understanding of networking concepts and storage management, including data partitioning, replication, and distributed file systems.

6. Cloud Integration

Knowledge of cloud-based Hadoop solutions and integration with cloud platforms like AWS, Azure, or Google Cloud.

7. Collaboration & Communication

Strong interpersonal skills to work effectively with cross-functional teams, providing technical support and clear communication with stakeholders.

8. Problem-Solving & Logical Thinking

Ability to troubleshoot complex technical issues, quickly diagnose root causes and implement effective solutions.