Data Specialist Jobs in Cave Spring, VA

About Us: LTIMindtree is a global technology consulting and digital solutions company that enables enterprises across industries to reimagine business models, accelerate innovation, and maximize growth by harnessing digital technologies. As a digital transformation partner to more than 700+ clients, LTIMindtree brings extensive domain and technology expertise to help drive superior competitive differentiation, customer experiences, and business outcomes in a converging world. Powered by nearly 90,000 talented and entrepreneurial professionals across more than 30 countries, LTIMindtree - a Larsen & Toubro Group company - combines the industry-acclaimed strengths of erstwhile Larsen and Toubro Infotech and Mindtree in solving the most complex business challenges and delivering transformation at scale. For more information, please visit ******************** Job Title: Databrick Administrator Location: Charlotte NC Duration: Fulltime Job Summary: Key Responsibilities: 1. Databricks Platform Administration Manage and maintain Databricks workspaces, clusters, and notebooks to ensure optimal performance and availability. Set up and configure Databricks Runtime Versions, High-Concurrency Clusters, and Auto-scaling policies for efficient compute utilization. Monitor cluster health, node utilization, and job execution performance for proactive troubleshooting. Implement Databricks best practices to optimize query execution and data processing speeds. 2. Security, Access Control & Governance Manage Databricks authentication and authorization (RBAC) using Azure AD, AWS IAM, or Google IAM. Enforce fine-grained access controls (Table ACLs, Unity Catalog, SCIM integration) for secure data access. Implement data encryption, tokenization, and masking strategies to comply with security policies. Ensure GDPR, HIPAA, and other compliance requirements are met for data security and privacy. 3. Cost Optimization & Performance Tuning Monitor Databricks cost usage and implement cost-control strategies (e.g., auto-termination, cluster rightsizing, spot instances). Optimize Spark jobs, partitions, caching strategies, and parallel execution for faster performance. Manage delta lake optimizations (Z-ordering, vacuum, compaction, and indexing) to improve query efficiency. Perform profiling and debugging of slow-running queries and jobs. 4. Automation & Infrastructure as Code (IaC) Automate Databricks workspace provisioning, cluster deployment, and job scheduling using Terraform or ARM templates. Implement CI/CD pipelines for Databricks notebooks, workflows, and integrations using Azure DevOps, GitHub Actions, or Jenkins. Develop APIs and scripts (Python, PowerShell, Bash) to automate Databricks administration tasks. Maintain Infrastructure-as-Code (IaC) templates to ensure repeatable and scalable Databricks configurations. 5. Integration & Data Connectivity Manage connectivity with external data sources, including Snowflake, Azure Data Lake (ADLS), AWS S3, Google BigQuery, SQL Server, and Oracle. Configure Databricks JDBC/ODBC drivers, API integrations, and partner connectors. Troubleshoot network, authentication, and API issues with cloud storage and external databases. Ensure proper networking setup (VNET peering, private link, firewall rules) for secure and reliable connectivity. 6. Job Scheduling & Workflow Orchestration Administer Databricks Job Scheduler and optimize notebook executions, streaming jobs, and workflows. Integrate Databricks with Airflow, Control-M, or Azure Data Factory (ADF) for workflow automation. Monitor long-running jobs, identify bottlenecks, and optimize execution timelines. 7. Monitoring & Troubleshooting Set up Databricks monitoring dashboards using Azure Monitor, AWS CloudWatch, Datadog, or Prometheus. Investigate job failures, performance degradation, and cluster errors using logs and metrics. Ensure continuous monitoring of Databricks workspace utilization and incident management. Required Skills & Experience: Technical Skills: 5+ years of experience in Databricks administration on Azure, AWS, or GCP. Deep expertise in Apache Spark tuning, cluster scaling, and job optimization. Strong Databricks security administration experience (RBAC, Unity Catalog, SCIM, OAuth). Hands-on experience with Terraform, ARM Templates, or CloudFormation for Infrastructure-as-Code (IaC). Proficiency in Python, SQL, and Bash/PowerShell scripting for automation. Experience with monitoring & logging tools (Datadog, CloudWatch, Prometheus, Azure Monitor). Strong knowledge of cost optimization techniques for cloud-based Databricks deployments. Familiarity with Delta Lake, Lakehouse architecture, and MLflow integration. Experience integrating Databricks with Snowflake, Azure Synapse, AWS Redshift, or Google BigQuery. Working knowledge of DataOps, CI/CD pipelines, and DevOps tools (Azure DevOps, GitHub, Jenkins). Soft Skills: Strong troubleshooting and problem-solving skills for critical system issues. Ability to work independently and collaborate across IT, security, and data engineering teams. Excellent communication skills for engaging with stakeholders and providing platform support. Strong documentation skills to maintain best practices, playbooks, and operational guides. Preferred Qualifications: Bachelor's/master's degree in computer science, Data Engineering, Cloud Computing, or related field. Databricks Certifications (Databricks Certified Data Engineer Associate, Databricks Certified Administrator). Experience working with large-scale distributed data processing environments. Knowledge of machine learning, MLflow, and Databricks AutoML (optional). Hands-on experience with multi-cloud Databricks deployments (Azure + AWS or GCP). Benefits/perks listed below may vary depending on the nature of your employment with LTIMindtree (“LTIM”): Benefits and Perks: Comprehensive Medical Plan Covering Medical, Dental, Vision Short Term and Long-Term Disability Coverage 401(k) Plan with Company match Life Insurance Vacation Time, Sick Leave, Paid Holidays Paid Paternity and Maternity Leave The range displayed on each job posting reflects the minimum and maximum salary target for the position across all US locations. Within the range, individual pay is determined by work location and job level and additional factors including job-related skills, experience, and relevant education or training. Depending on the position offered, other forms of compensation may be provided as part of overall compensation like an annual performance-based bonus, sales incentive pay and other forms of bonus or variable compensation. Disclaimer: The compensation and benefits information provided herein is accurate as of the date of this posting. LTIMindtree is an equal opportunity employer that is committed to diversity in the workplace. Our employment decisions are made without regard to race, color, creed, religion, sex (including pregnancy, childbirth or related medical conditions), gender identity or expression, national origin, ancestry, age, family-care status, veteran status, marital status, civil union status, domestic partnership status, military service, handicap or disability or history of handicap or disability, genetic information, atypical hereditary cellular or blood trait, union affiliation, affectional or sexual orientation or preference, or any other characteristic protected by applicable federal, state, or local law, except where such considerations are bona fide occupational qualifications permitted by law. Safe return to office: In order to comply with LTIMindtree' s company COVID-19 vaccine mandate, candidates must be able to provide proof of full vaccination against COVID-19 before or by the date of hire. Alternatively, one may submit a request for reasonable accommodation from LTIMindtree's COVID-19 vaccination mandate for approval, in accordance with applicable state and federal law, by the date of hire. Any request is subject to review through LTIMindtree's applicable processes.