Job description :
Function Description
Data Engineering is the application of engineering to design, develop, implement, test, deploy, monitor and maintain the delivery of data in a systematic method.
The roles in this function will cover all primary development and maintenance activities across all technology functions to ensure that we deliver data with high quality for our users, applications, and services and to understand customer needs.
The capabilities include, but are not limited to analysis, design, coding, engineering, testing, debugging, standards, methods, tools analysis, documentation, research and development, maintenance, new development, operations and delivery.
With every role in the company, each position has a requirement for building quality into every output.
This also includes evaluating new tools, new techniques, strategies; Automation of common tasks; build of common utilities to drive organizational efficiency with a passion around technology and solutions and influence of thought and leadership on future capabilities and opportunities to apply technology in new and innovative ways.
Analyzes current business practices, processes and procedures as well as identifying future business opportunities for leveraging data for analytics, reporting, machine learning and other data intensive activities.
Manages relationships with software and hardware vendors to understand the potential architectural impact of different vendor strategies and data acquisition.
write SQL or other data markup scripting to help support development of Analytics and Applications that build on top of data.
Data Engineering Core Tasks
Accountable for data engineering lifecycle including research, proof of concepts, architecture, design, development, test, deployment and maintenance.
Design, develop, implement and run cross-domain, modular, flexible, scalable, secure, reliable and quality data solutions that transform data for meaningful analyses and analytics while ensuring operability. Design, develop, implement and run data solutions that improve data efficiency, reliability and quality, and are performant by design.
Layer in instrumentation in the development process so that data pipelines can be monitored. Measurements are used to detect internal problems before they result into user visible outages or data quality issues.
Build processes and diagnostics tools to troubleshoot, maintain and optimize solutions and respond to customer and production issues
Embrace continuous learning of engineering practices to ensure industry best practices and technology adoption, including DevOps, Cloud and Agile thinking
Tech debt reduction/ Tech transformation including Open source adoption, Cloud adoption, HCP assessment and adoption
Contribution to client Inner source / industry community and strive to reuse and share components wherever possible across organization.
Maintain high quality documentation of data definitions, transformations, and processes to ensure data governance and security.
DHIRENDRA DEO
Appian Infotech Inc.
Contact No : 276) 910-0132
Email id : dhirendra.d@appianinfotech.com
WebSite : (link removed)
