Download to read offline
Uploaded byBizTalk360
One repo for all with Azure DevOps
The presentation by Kamil Mrzygłód on Azure DevOps discusses implementing continuous integration and continuous delivery (CI/CD) through the use of monorepos. It covers the characteristics and advantages of monorepos, various pipeline types (classic vs. YAML), and best practices for managing repositories and permissions. Additionally, it touches on pipeline triggers, templates, and policies linked to deployments.
More Related Content
Similar to One repo for all with Azure DevOps
![[JAZUG Tohoku Azure DevOps] Azure DevOps](https://cdn.slidesharecdn.com/ss_thumbnails/20181130jazugsendaiazuredevops-181201031328-thumbnail.jpg?width=640&height=640&fit=bounds)
![Support, Monitoring, Continuous Improvement & Scaling Agentic Automation [3/3]](https://cdn.slidesharecdn.com/ss_thumbnails/agenticcommunityseries-day3-cfd-251120170304-ddef8112-thumbnail.jpg?width=640&height=640&fit=bounds)
One repo for all with Azure DevOps
- 1. One repo forall with Azure DevOps A presentation by Kamil Mrzygłód
- 2. About me Senior SoftwareDeveloper @ Demant Located Warsaw, Poland Trainer & Consultant Books author Microsoft Azure MVP
- 3. Agenda CI, CD,Azure DevOps A repository Monorepo Building a monorepo Classic vs YAML pipelines YAML pipelines features Policies, approvals, environments Questions! What’s for today?
- 4.
- 7. Solutions for DevOps Leverage thebest practices: CI/CD Agile Version control Infrastructure as code Configuration management Monitoring
- 9.
- 10.
- 12. Arepo Source oftruth for our codebase Source for our build pipelines History of a project Auditing tool Ultimate value of a project
- 13.
- 14.
- 15. Comparison Common andstandard pattern, especially for microservices Handled individually by a dev team or a dedicated person Scoped to a single project, not need to granular access to various components Easy to Build Represents only a tiny bit of a system Sharing functionality actually involves additional component to make it functional Aggregates multiple features of a common application(platform) Requires a dedicated team managing a repository Might be tricky to actually prevent someone from changing a particular feature Represents the whole system but often involves a multi-step build pipeline Easy to share you work with others
- 16.
- 17.
- 18.
- 21.
- 22.
- 23. Multiple levels You can decidehow a team or a group can access e.g. a repository.
- 24.
- 25. What’s a pipeline? A setof stages, jobs and steps, which build, test and/or deploy your application. Can cover a single service or the whole platform.
- 26. Pipeline in Azure DevOps You haveaccess to the full log to debug and analyze how pipeline is run.
- 27.
- 30.
- 33. YAML vs Classic pipelines YAML Writtenin YAML Easy to version Easy to share Consolidate whole application lifecycle Can be easily extended using shared templates Classic Developed with GUI(with underlying JSON representation) Versioning is really tricky Sharing requires import/export functionality Based on separate build/release pipelines Can be extended via Task Groups
- 34. How to enhancea monorepo with YAML pipelines?
- 35.
- 36. Trigger types Push trigger –based on branch PR trigger – triggers a PR build Scheduled trigger – based on a schedule
- 37.
- 38.
- 39.
- 40. Stage template Reuse a wholestage functionality(highest level)
- 41. Stage template Reuse a wholestage functionality(highest level)
- 42. Job template In a single/multiplestages reuse a particular functionality(e.g. installing dependencies)
- 43. Job template In a single/multiplestages reuse a particular functionality(e.g. building an application)
- 44. Steps template Reuse only asubset of functionalities(e.g. installing dependencies & testing)
- 45. Steps template Reuse only asubset of functionalities(e.g. installing dependencies & testing)
- 46.
- 47. Policies Currently scoped onlyto container images and Kubernetes environments. Written in Rego.
- 48. Environments Environments links informationabout deployment with steps performed while running the pipeline. They allow also for a resource linking(Kubernetes namespace, VM).
- 49. Approvals Environments links informationabout deployment with steps performed while running the pipeline. They allow also for a resource linking(Kubernetes namespace, VM).
- 50.
- 51.
- 52.
- 53.
Editor's Notes
- #2 Thanks guys for inviting me to this webinar. I hope some of you are interested in topics like application lifecycle, devops, application development. This presentation is about 1hour talk, during which I’ll try to introduce you to Azure DevOps, repository modeling and multi-team development.
- #3 I’ll share social media links at the very end of this presentation in case you’d like to get in touch and chat a little bit about development and Azure.
- #4 High level agenda which is a basic journey from an idea how your repository may look like, what a tool to choose and how to Build and manage its lifecycle. We’ll Focus on one of the newest feature of Azure DevOps which are YAML pipelines. The last part of the agenda dives deeper into YAML pipelines, so you’ll have a Chance to get some insights what is possible currently to get things working as expected and per your requirements. I’ve some time reserved for answering questions so if anything is unclear, please keep them so I can answer later.
- #5 A Microsoft tool to Build applications, deploy them and collaborate with teammates. Previously named VSTS(Visual Studio Team Services), rebranded several months ago following Focus on cloud. Includes issue tracker, repository management, Build server and release manager. Offers some extra features like package/artifacts manager and wiki. Integrates with GitHub and can be used for free in smaller projects. I personally use it for my projects to achieve a coherent development environment.
- #6 With end-to-end solutions on Azure, teams can implement DevOps practices in each of the application lifecycle phases: plan, develop, deliver, and operate. These DevOps technologies, combined with people and processes, enable teams to continually provide value to customers. This slide presents key takeways from Azure DevOps – with that tool, it’s easy to manage the whole application lifecycle – from plan and development, through delivery to operations. It’s a unified and complete environment compliant with even more strict enterprise rulesets. My current company is a medical company, which has to follow some serious requirements to be compliant with regulations from such organizations like FDA in US and yet we embraced and mastered Azure DevOps as our tool-of-choice for cloud-based services and multi-platform development.
- #7 I already mentioned that but to make that clear for you – you can integrate GitHub account with Azure DevOps, still those are separate products and you can leverage Azure DevOps for its Build and realase features while keeping the hear of collaboration still on GitHub(like repository and issue tracking). It’s possible to import a GH repository to DevOps but it’s not mandatory to keep the integration working.
- #8 DevOps solution that meets your needs with built-in Azure services such as Azure DevOps. Likewise, put together a combination open-source DevOps tools to match your unique workflow, then seamlessly integrate them on Azure. Whether your app uses virtual machines, web apps, or Kubernetes, implement DevOps practices like continuous integration and continuous delivery (CI/CD), infrastructure as code, and continuous monitoring with Azure and the DevOps toolchain of your choice.
- #9 In case you are familiar with other tools for ALM, you’ve problably heard of applications like TeamCity or Jenkins. I personally used and loved tchem both, but as long as I focused on cloud development, Azure DevOps seemed to me like a more natural choice. It’s possible to integrate those with Azure, it’s just not as natural to me as using DevOps. Please don’t take my words as recommendation as personally I might be slightly opinionated – at the end, it’s a matter of choosing what fits your setup the most.
- #10 With that short introduction we’ve finally came to our first question related to today’s topic, which is:
- #11 I believe that most of you work with repositiories like git, SVN, mercurial or other, but I’d like to keep the story of this talk in a way, that I describe as many terms as possible not to favour anyone.
- #12 So a repository is a history of a project, application or platform. Simply put, it doesn’t matter what is put into a repository – any change is recorded and decorated with appriopriate metadata, which differ depending on the repository engine. I mentioned different repository engines like SVN or git, which offer different features and functioinality, but the key mechanism is the same – to keep the project history in a durable and resilient state.
- #13 We’ve just forged a more elaborate definitione, but one can quickly sum up a repository as the following „being”:
- #14 With a repository explained the best I could, we can go deeper into the topic and finally decide what is a special kind of repository called a monorepo. A typical repository contains a single project with a Limited numer of collaborators(though one can underestimate the power of OSS – e.g. take a look and k8s repository on GitHub which is backed by over 24 hundred individual contributors). The key feature if a single repository is the fact, that it limits what is built from it – it’s single feature rather than a platform.
- #15 This is why a concept of monorepo was made – as in large projects we often create a single application from multiple features, instead of merging all the different repos with different artifacts, we could keep everything in a signle place and divide it logically and using policies while enhancing collaborating capabiltiies. Remember to avoid to compare a monorepo with a monolith as a monolith in fact can be created with a monorepo, but monorepo can be used also for microservices deployment(what is true for one of my projects).
- #16 Here’s a short summary – on the left with have a dedicated repository while on the right monorepo.
- #17 How a very basic structure of a monorepo would look like – here’s an example. As you can see, we have logically divided shared and individual project components. The important thing here is keeping things separated on the lowest level, this is filesystem. This will help you in monorepo management as e.g. you could assign different sets of permissions to different teams. This is true for one of my projects where `shared` capabiltiies can be changed by everyone, but cannot be pushed to the master until validated by the core team.
- #18 We talked a little bit about a monorepository, let’s check how one can start it with Azure DevOps. The point here is to show you how leveraing various Azure DevOps features you can initialize and work with a monorepo if you find the concept interesting and suitable for you.
- #19 The very first thing – you should create a team! Team in Azure DevOps are like groups – they aggregate all the user catalogue members, which deliver the same funtionality or share the same role. This is a must have as handling permissions for each user individually will be a real pain and you don’t want to do this. This enhances and also speeds up the proces of onboarding new members of teams – instead of remembering where to put tchem and what permissions assign, you can just add the to a DevOps team – all the assignments will be propagated automatically.
- #20 Here’s the basic look of teams in our monorepo. As you can see, each team represents a separate project in your domain, though it could be a feature or a phisical team sitting in your office – the concept is the same, you have to adjust it to the reality you live in. Let’s check how one can create a team in Azure DevOps:
- #21 Creating a team is a matter of just a few clicks – you start with a New Team, then you have to provide a team name. The description is optionall, but will be helpful if a name isn’t enough to identify it. Each team can represent a particular subset of permissions and to be honest the feature would not be complete with proper access management.
- #22 Working with a team will never be complete without assigning a proper set of permissions to it. Permissions are used to decide what a team or a team member is allowed to do – this can be more general action like modifying the project in Azure DevOps, accessing/mutating a repository or even editing a pipeline. Defining a granular and useful set of permissions is for sure something, which will take some time util the whole project is set solid, but getting some initial idea will be really important to get everything correctly. Like we mentioned earlier with separated shared and project spaces inside your repository, thinking about permissions Ahead will save you much time.
- #23 Here you have an examplke of how permissions may be set on a team level. This particular area in Azure DevOps covers high-level permissions like access to particular features of Azure DevOps(Boards, Analytics) and ability to work with a project. For monorepo you probably introduce a core team, which will handle most high-level operations, so denying most of the actions here looks like a good idea for common project teams. Here you can see the strength of leveraging team-oriented access management – with a single click you can decide that all the team members can e.g. delete work items.
- #24 Team in Azure DevOps(and users) can be grouped into groups, what makes management even easier. Let’s model a following scenario – you have a monorepository containing several teams. You’d like to assign a permission to all the teams with a single click. While it would possible with team only(by individually assigning a permission), you can group tchem into a single group called e.g. ‚Platform Teams’ and then decide, that the whole group cannot bypass polciies when pushing code(so they cannot break the current setup). This step is essential to get the best from a monorepo – everything is structured and covered by policies so multiple teams can work with each other with no or little interferences.
- #25 Okay, let’s now Focus on the next core functionality of Azure DevOps, which will help us work as a platform on delivering our functionalities. In Azure DevOps, whole application lifecycle is described using various stages, jobs and steps which are called pipelines. When delivering monorepo, each project/team will develop its own pipeline and such a team is responsible for maintaining it. We can limit what a team member can do with a pipeline(for example we can limit ability to delete it while allowing editing), but the core shape of a pipeline is mainly developers responsibility. This is also a pure example of DevOps culture when devs are taking care of delivering a deployment procedurę, which can be used by release engineers.
- #26 How one can sum up that description? Well, we can define to key features which identify a pipeline. [READ] You probably Wonder how a pipeline may look like in Azure DevOps. Well, here’s an example of a very simple application, which was built by an agent hosted by DevOps.
- #27 As you can see, this particular pipeline contains a single job, which is built with various steps. Each step identifies a single action taken to Build, test or deploy an application. We can either download package, run a Build script, test assemblies or push artifacts to our server. Of course we are not Limited only to those action – Azure DevOps offers many different steps, which cover platforms like .NET, Docker, Kubernetes, Java, JS, Pyton and so on. When a pipeline is executed on an agent, we get a constant view of what happens – as you can see on the slide, all the output generated by various commands is saved and available to you to verify the proces. Remember that for monorepo those pipelines will be executed individually and if your platform is on a hot path of development(what implies multiple Builds run in the same moment), you may need to get extra agents which can handle the pressure.
- #28 In Azure DevOps we have two different types of pipelines – Classic and YAML. Classic pipelines are an original feature of Azure DevOps, which can be developed using a GUI. YAML pipelines are quite a new feature, which introduce a new way of developing application lifecycle by writing pipelines in YAML. This introduces some interesting possibilities, which we’ll cover later, but for now let’s see how both types of pipelines look like and what are the key differences.
- #29 A Classic pipeline consists of various steps added manually, which are configured according to your application. Each pipeline consists of phases run on a preconfiogured agent(what determines things like OS or installed software). Based on the steps configured, each agent will be tested against demands which it has to cover in order to be able to run the pipeline(like availaibility of NodeJS or .NET Framework). The main downside of such pipelines is the fact, that versioning, development and testing are really tricky – while such a definitione can be exported as JSON, doing so is not natively supported and JSON development is not ideal in most cases. To test a change you have to duplicate the pipeline and manually reconfigure what takes Times. This makes it difficult to use in monorepo when all the development is based on flexibility of Azure DevOps to bend to a particular team needs while still satisfying more general policies.
- #30 Here you can see the whole Classic pipeline.
- #31 How about YAML pipelines? AS i mentioned earlier, it’s quite a new feature and still in active development., It exchanges GUI based envuironemtn ing favour of writing the pipeliens from scratch using YAML. YAML is not ideal, some people rather dislike it but it’s still less verbose than JSON, what makes it a fine choice for developing application development pipelines. You may Wonder how a simply pipeline looks like – here’s an example:
- #32 Here you can see a screen from VS Code with a very basic pipeline ready to rock. It contains a few areas separated from each other – triggers, pools and steps. As it’s really simply it misses some more advanced structures(like stages or environemtns), but it’s a great base for getting started. With tools like VS Code you can get syntax support or even code snippets to use, to using this editor with YAML pipelines makes things much smoother. Of course schema for YAML pipelines is officially available and is a must have when developing tchem.
- #33 When using Azure DevOps with YAML pipelines you will also get editor for your use, so you can change the pipeline directly in your DevOps project. It also offers syntax highlighring and help with snippets, so in case of quick fixed or testing things out, it’s a really nice addition to the twhole environemnt.
- #34 Before we dive in, here’s a quick comparison of YAML and Classic pipelines. While I believe developing a monorepo will be also possible with Classic pipelines, it’ll be much easier when working with Azure DevOps if you select YAML pipelines, mostly due to sharing capabilities.
- #36 We’ll start with triggers. Each(ven Classic) pipeline requires a trigger to work. Triggers make your piepline run and introduce a logic into the project. You want to Build the whole platform? Trigger a Build when a new commit comes to the master branch. Feature builds? Add a wildcard and a project name incorporated into the branch name. There’re countless capabilities but this is required to make a monorepo working(nothing’s worse than triggering all the Builds for all the projects with a single commit).
- #37 There’re actually three different types of triggers in Azure DevOps. While PR triggers work only for GitHub/Bitbucket, the rest two can be easily used in your pipelines. The most common type is of course push trigger and you will use it the most while keeping the scheduled trigger for most specific scenarios. Personally I use scheduled triggers for running pipelines, which are not related to the platform or the project itself(like cleaning some artifacts, ensuring that policies are satsfied or veryfing stuff). Each trigger has its own definitione in YAML:
- #38 It’s important that making path filters working require specifyuing bracnshes al.;so.
- #40 Templates are what makes YAML pipelines so good in monorepos – it’s the main sharing capability between projects which allows easy separation of concerns. You need to share the whole stage, job, steps between projects? Just introduce a single YAML file and enforce all terams reference it. Templates lower readdability of pieplines a little bit but the cost is worth it – no need to repeat work in multiple places, introducing changes is much simpler, you can even share variables to be used across different projects.