This blog is dedicated to Coral Project, I will describe the various phases and challenges of the project from the DevOps perspective.Its a Mozilla Foundation project, in collaboration with the New York Times and the Washington Post.
We are creating open source tools and practices for newsrooms of all sizes, to build better communities around their journalism. That means better comment boxes, better user-generated content modules, and a whole lot more.
As a Mozilla project, the user's control over their privacy and identity will be paramount throughout.
My name is Eugene de Fikh, I am a DevOps engineer who has been with a Coral Project starting mid December 2015. I am in charge of the infrastructure and automation using todays DevOps principles. I have been a systems engineer for the past 15 years , in the last 5 years I have been working on automating AWS infrastructures for various startups.
Current setup is done in AWS console using VPC subnets. I have assigned private and public subnet , creating a three tier architecture design. We allow web servers to talk to the outside world and to middle tier servers. Middle tier is allowed to make direct connection to database. Database servers are not accessible from outside. In addition we use OpenVPN server to allow access to developers per allowed groups. We will be implementing OpenLdap to manage ssh keys and rotate users logins on regularly scheduled basis.Backend is run by Mongo db 3.2 cluster , with a primary and 2 secondary servers.
Middle tier is Pillar application and front end is Xenia web servers. We choose to setup Dev/Stage/Prod environments and automate server and configuration management using Puppet server. Puppet server manages common server deployments and setup of user, ssh keys, configuration files for application use, server partitioning, etc.We choose to create custom AMIs with 3 types of servers, web , middle tier and database servers. Each type of server can reside in one of the 3 environments ( Dev/Stage/Prod).
Our CI server is Jenkins that resides in private subnet and is allowed to deploy using ssh keys to middle tier and web servers for stage only at this time. We have 2 jobs in Jenkins, one to create the build and generate artifacts that then get deployed by a second CI job to each of the designated servers ( web / middle tier). We automated build process to check every 30 minutes with github for an updated master branch. If changes are detected new build is generated and automatically tested for errors and deployed to staging server. if a job fails an email is sent to the team letting them know of a failure. In the future posts I will describe deployment in more details and also will talk about monitoring and log management.