Keyman will come to Japan again this year from GitHub headquarters. Special release of the latest roadmap

This year again, Mr. Paterson, a solution engineer from GitHub, Inc. head office, came to Japan. I was in charge of the opening session, and talked about the major investment areas that GitHub is currently focusing on, the roadmap for the latest technology, and our thoughts on Inner Source.

As soon as the preparations for the new features that will be released in the future are completed, we will be able to tell you about them one by one, so please look forward to it.

"Utilization of GHE in intra-mart development"

NTT DATA INTRAMART CORPORATION. Mr. Takuya Akuzawa

NTTデータイントラマートは、Webシステム基盤を構築するためのパッケージソフトウェア「intra-mart」の開発を始めとして、パートナーを通じての販売、導入、サービス・コンサルティングの提供を行い、1998年から現在まで、約6000社への導入実績があります。

The company's development headquarters is working to improve the development environment for GitHub Enterprise and streamline release work. The package development team is about 70 people, and the development cycle is 3 times a year with 4-month intervals, and patches and documentation are updated every month. Currently, all source management is done with GitHub Enterprise, the number of repositories exceeds 1000, and the build mechanism is all automated using Jenkins jobs. Akuzawa explained that the number of commits is more than 190,000, and the entire library exceeds 11.45 million lines.

The company used Subversion before introducing GitHub Enterprise. At that time, the only way to get reviews was to commit directly to the main development stream, and there was a possibility that incomplete code that didn't work would be committed. In addition, Mr. Akuzawa confesses that there were problems such as failures in the CI environment even though the build was successful locally, and frequent conflicts between workers during busy periods, making parallel development difficult.

At the end of 2016, we decided to use GitHub Enterprise for source code management, which led us to migrate most of our build environment from internal physical servers to AWS. For example, when the developer pushes the source code, the Hook flies, the Jenkins job automatically executes, and the Docker image is pulled. Using AWS's ECS, the build and test are performed cleanly in a Docker container on EC2, and finally deployed to complete the addition of functions.

NTT DATA INTRAMART GitHub Enterprise and CI environment

Having operated GitHub Enterprise for two years and developed a total of seven times, branch operations have now become easier, and it is now possible to easily commit from multiple feature branches without affecting the main development stream. In addition, the Status Check function confirms the presence or absence of reviews, Jenkins build results, warnings from a unique checker tool, etc., to ensure the quality of the deliverables to be merged. Greatly improved development efficiency. In addition, the pull request function increases review opportunities and reduces review omissions. Parallel development between seasons and between workers has become much easier, which has led to improved development quality.

In addition, Mr. Akuzawa disclosed API usage methods such as Redmine linkage, Coverity (static analysis) linkage, and simultaneous automatic merging using Python scripts.

"About GHE introduction and permeation process at contracted web development site"

FORK CORPORATION., Ltd. Board Director Tomohiro Onuma

Fork is a company that provides one-stop solutions from design to system development, focusing on campaign sites and brand site construction related to corporate promotion. The company used to use Subversion, and its repositories were managed in a distributed state on multiple internal servers and external production companies. Looking back, Mr. Onuma recalls that the challenges included modernizing the development environment to meet industry standards, developing a deployment method that avoids problems at the time of publication, and cultivating a team development culture through daily reviews.

In April 2016, the GitHub Enterprise implementation project started. In order to spread it within the company, we created a private repository on GitHub.com and migrated to GitHub + Jekyll (static site generation tool) at the timing of renewing our site. As a result, public relations and human resources were able to get used to GitHub. In addition, since we hire about 10 new graduates every year, we have created an environment that makes it easy for them to enter the project by submitting HTML and CSS training results to GitHub Enterprise.

A fork that created an opportunity to get used to GitHub on our site

The company often has many projects that will be operated for a long time after creating a website, and the issue was when to migrate existing projects. I moved them step by step.

After the introduction of GitHub Enterprise, highly sensitive employees actively led the introduction, and it is said that engineers' dissatisfaction such as insufficient information in the issue pull request template was greatly reduced.

About two and a half years have passed since its introduction, and now 110 people, mainly engineers, are using it, the number of organizations has increased to 16, and the number of repositories has increased to about 1,000. Mr. Onuma said that in the future, he would like to make code reviews a habit on a daily basis and consider the participation of external partners in order to revitalize collaboration.

"SIOS InnerSource and GitHub"

Mr. Masataka Kurihara, Managing Executive Officer, SIOS Technology, Inc.

In the past, Mr. Kurihara launched a user forum dedicated to Delphi, an integrated development environment, and has also served as president of Glugent Co., Ltd. and President of the NPO Seasar Foundation. He is known as a front-line OSS developer and Java engineer, such as working on Mayaa development. Currently, he is active as Managing Executive Officer of SIOS Technology, which provides a wide range of services with OSS and Java as core technologies. The company also has a base in South Carolina, USA, and is characterized by product development through close international cooperation between Japan and the United States.

In the previous development environment, small organizations of about 10 people were scattered in a silo state, so the repository and issue management were handled independently by the team, and the languages, libraries, and clouds used were both paid and free. It is said that there was Mr. Kurihara recalled that he lacked a sense of unity as a technology company, such as asking for help from the community rather than within the company when something went wrong.

The key points in selecting and configuring an ALM (development management environment) tool were 1) that the tool should be popular with the engineers who use it, 2) emphasize security and compliance, and 3) There were three things: improving the skills of engineers and making it possible to drive internal and external communication from the software side.

As a result of consideration, we adopted GitHub Enterprise in the fall of 2018 and linked it with Slack, JIRA, etc. As a result, while promoting the spread of social coding methods and dissemination of technical knowledge inside and outside the company, we have realized the automation of information sharing through chat and the work style of business execution. SAML (single sign-on) / SCIM (ID information linkage) and BOT integrated ALM tools on the cloud.

In addition, AWS Lambda (a serverless programming code execution service) is used to daily back up the AMI (Amazon Machine Image) of the instance on which GitHub Enterprise is running, and Amazon EC2 is used for VPC-peering (Amazon We use github/backup-utils (a genuine GitHub backup/restore tool) to connect with overseas bases via a network connection that enables routing of traffic between Virtual Private Clouds.

System configuration of SIOS Technology centered on GitHub Enterprise

Several months have passed since the start of operation of GitHub Enterprise, and currently there are about 70 accounts in sells hosting operation, but it is said that the scope of operation will be expanded in the future. He also said that in order to migrate from existing issue management tools and improve latency in the Japan-US development system, installation of instances in the US region and compliance issues (ISMS support, intellectual property protection, etc.) are issues.

Next, Mr. Kurihara mentioned the value of Inner Source. Inner Source is a philosophy that applies OSS development style and culture to in-house software development, breaking down internal siled bureaucracy and encouraging developers to collaborate autonomously. It is also used in the sense of sharing existing knowledge across multiple lines. The company is working to utilize the resources (knowledge, experience, and deliverables) utilized by other departments through Inner Source in parallel development projects.

Mr. Kurihara points out that now that Pull Request has been developed by GitHub and GitHub.com has become a central repository at the development language level, well-known companies are creating OSS with huge capital investment. He said that it is important not only to use OSS, but also to create it.

"Development of CI/CD environment with Github Enterprise and Drone.io"

Internet Initiative Japan Inc. Cloud Headquarters
Mr. Kazuki Hamazaki, Public Resource Section 1, Cloud Service Department 1

Internet Initiative Japan (IIJ) is a stand-alone company with about 2,000 employees, and offers a wide range of services, including cloud, security, networks, data centers, SI, and mobile, and 70% of its employees are engineers. GitHub Enterprise currently has 470 seats, about 8000 repositories, and about 300 organizations. Mr. Hamasaki is in charge of the development of the public cloud (IIJ GIO) and has also been the administrator of GitHub Enterprise since 2017. After working with Subversion and GitLab/Gitorious (experimental), the company introduced GitHub Enterprise in 2013. After that, we introduced Drone.io in 2014 and Microsoft Teams (chat) in 2018.

IIJ provides a wide variety of services such as SaaS, IaaS, embedded, and line, so there is no unified development flow. For that reason, they say that they are preparing general-purpose tools as much as possible, so that development languages and tools can be freely selected for each project.

The company introduced GitHub Enterprise one of the fastest in Japan, but Mr. Hamasaki recalls that at the beginning of the introduction, it was still difficult to link with the CI/CD service, and it was a trial and error process. It was Drone.io that appeared in such a situation. Drone.io is a Docker-based CI that is highly compatible with GitHub Enterprise, and the point is that it can automatically run tests in a closed environment inside a Docker container. In order to use Drone.io conveniently, we have provided a set of Docker Private Registry for storing images and Docker Registry Frontend for managing registered images as a set so that they can be used without application.

IIJ Drone.io management screen example

Since their introduction, GitHub Enterprise and Drone.io have been actively used as a place to publish the source code of browser extensions for internal use and API-wrapped libraries for use as internal tools. It is said that a virtuous circle has been created in which a pull request is sent by an engineer who has no acquaintance in the company, someone voluntarily makes a correction, it is advertised on Confluence (Wiki), and it spreads throughout the company. In addition, the support department, which does not write code, seems to be using it for building and publishing manuals, reviewing announcements for customers by pull request, and checking Japanese by CI.

However, as a result of promoting the use without applying, there were many accounts that were not used much even though they were registered. Due to the maintenance cost, we used Apache AirFlow to automate the suspension work. Also, when a user leaves the company, the access key issued to that user for linking with other apps becomes invalid, and GitHub integration becomes impossible. Developed and resolved.

In the future, we will create educational materials including CI/CD to explain how to use GitHub Enterprise, especially for people who do not know how Git works, and for departments that do not have a culture of version control and ticket management. I said I was planning to go.

"JFrog Artifactory introduction case study"

Stackwide Reliability Section, Cloud Platform Department, Rakuten Group, Inc.
Mr. Takaaki Furukawa, EC Infrastructure Group

In December 2018, Macnica entered into a distributor agreement with JFrog Ltd., headquartered in Israel, and began providing the DevOps solution "JFrog Enterprise, Enterprise+." JFrog's solutions have been implemented by over 4,600 companies around the world, and support continuous and efficient implementation of a series of processes from software development to release. Specifically, "JFrog Artifactory" (manages artifacts such as software, libraries, and Docker images), "JFrog Xray" (scans artifacts for vulnerabilities and detects OSS license violations), and "JFrog Distribution" (software It consists of a group of functions such as "JFrog Mission Control" (a management console that can centrally manage the settings, monitoring, operation and maintenance of JFrog products), and some solutions are also available individually.

This time, Mr. Furukawa from Rakuten introduced the necessity of Artifact Registry and an introduction case of JFrog Artifactory (universal artifact repository manager tool).

Mr. Furukawa explains that the Artifact Registry is a place to store the artifacts of software development, and is indispensable for realizing CI/CD. Specifically, when committing changes to GitHub in software development, when GitHub kicks the CI tool with a webhook, etc., the artifact (Artifact) is built in the CI tool, and the destination to store the result is Artifact It becomes Registry.
Artifact Registry has two main advantages. One is dependency management. Since artifacts with dependencies can be handled statically, reproducibility of builds is guaranteed, and the problem of "it works on my device but not on other devices" can be prevented. It also helps keep your build pipeline simple by ensuring build consistency.
Another is security and compliance. By saving activity history such as who pushed and when, authentication and authorization mechanisms are supported.

Rakuten chose JFrog Artifactory to implement the Artifact Registry. Rakuten's development organizational structure is divided into three layers: the development group, the operation group, and the infrastructure group. Before the introduction of JFrog Artifactory, the teams under Mr. Furukawa's control (operations group, infrastructure group) managed private repositories for server administrators. On the other hand, in the development group, each team was using different artifact repository packages on separate servers.

As a result, in addition to increasing the operational cost of unnecessary server resources and manpower, there were also security problems such as the occurrence of stray repositories and neglect of authentication/authorization operations and history management.

In the spring of 2018, the development team, which was already using JFrog Artifactory, asked me to take over the operation, which led to an organization-wide implementation. Regarding the reasons for standardizing JFrog Artifactory, Mr. Furukawa said that 1) it is a de facto standard worldwide, 2) it supports a wide range of package types, 3) it can support on-premises and public clouds, and 4) it is easy to charge. I mentioned that it is a product unit rather than a user unit.

Rakuten Introduces JFrog Artifactory to Centralize Deliverable Management

With the introduction of JFrog Artifactory, it was possible to centralize conventional artifact management, which reduced costs. It is said that cross-organizational collaboration is becoming active, such as team B using a library developed by team A.

However, during the operation of JFrog Artifactory, the number of users increased too smoothly, and Mr. Furukawa said that he was troubled by the frequent requests for repository creation. , I explained that I adopted "Policy as Code with GitOps", which manages policies such as repositories with GitHub.

In the future, we will incorporate the security perspective into CI/CD and promote DevSecOps, which aims to develop safe and secure software by turning feedback loops faster.

"Continuous integration starting with CircleCI"

CircleCI Japan Solutions Engineer Noboru Kurai

CircleCI was founded in San Francisco in 2011 to provide software developers with services to deliver better code faster. We currently have approximately 200 employees worldwide. CircleCI Japan was launched in June 2018. Kurumai joined as a Solutions Engineer in October.

At the beginning of the session, Mr. Kurumai explained how GitHub and CircleCI work together and how issues and pull requests can be displayed and utilized for those who are not familiar with CircleCI. It's also easy to get started, and CircleCI operates as an OAuth application for GitHub, so if you're logged in, you're done setting up linkage with GitHub. The authority will be transferred to CircleCI and the list of repositories will be displayed. After that, when you start the setup project, the sample will be displayed, so after adding the CircleCI configuration file (config.yml) to the repository, you can start building from the CircleCI screen using the Start Building function.

View pull requests in CircleCI UI

Next, Mr. Kurai explained three features for speeding up builds. The first is workflow. In the flow of checkout → solve library dependencies → build → test → deploy, we break down the build settings (jobs) as much as possible, define dependencies between jobs and perform parallel processing. The second is cache. In addition to reusing persistent data used by the same job in which the workflow is executed repeatedly, data is shared between different jobs within the same workflow. The third is parallel processing. For example, if you run 10 tests in 4 parallels, calculate how to divide the tests from the previous test results to equalize the time in 4 containers.
In addition, it is also possible to handle confidential information, such as login password for database connection and access key to the cloud, confidential information that you do not want to disclose only to some project members can be managed by CircleCI and the build is run. is possible.

So why is CircleCI capable of speeding up? Kurumai explained 1) how long it takes for code to be deployed after being committed, 2) how long it takes for a CI build, 3) how long it takes for a CI build to start, 4) how long the master branch is broken, and 5) He explained that this is because it can reduce the time required for things other than development, such as tool maintenance.
Mr. Kurai emphasized that CircleCI can be easily linked with GitHub, that developers can intuitively use it, and that it has functions to run CI builds at high speed.

"Introducing Sider, a code review support service linked to GitHub Enterprise that supports information sharing and growth of development teams"

Mr. Koichiro Sumi, Representative Board Director, Sider Inc.

Sider is a GitHub-linked code review support service that supports information sharing and growth of development teams. We officially released Sider for GitHub Enterprise in February 2019.

At the beginning of the session, Mr. Sumi said that although areas that can be automated through GitHub, CI/CD, infrastructure coding, and hardware improvements are definitely speeding up, there is still room for improvement in the coding environment where humans are involved, especially in the area of code reviews. I expressed concern that I was late. A lot of money and resources are spent on code review, 1) Inconsistent levels of understanding and proficiency among teams due to the increase in personal documents, 2) Waste of waiting time for review, 3) Repeated review of the same comments, 4) ) It is said that omissions and omissions caused by human error are accumulating day by day.

Sider is characterized by suggesting items to check on implementation instead of pointing out

Sider has its own analysis machine (published as OSS), and project-specific rules and historical background can be easily described in its configuration file (YAML file). This makes it possible to easily automate code reviews and prevent omissions and omissions. In addition, Mr. Kado emphasized that the feature is that it "suggests" as a confirmation item for implementation, instead of "pointing out" like a general analysis tool. It is automatically detected at the time of GitHub Enterprise Pull Request, and the inspection result (code review) is notified within about 1 to 3 minutes. Based on that, correct it if necessary and push it again, or close it to complete.

In addition, static analysis by linter is also supported. If you sign up and add a repository without writing rules, a general linter will start checking in conjunction with GitHub Enterprise Pull Requests, unifying style and best practices according to language with an optimal UI. Mr. Kado explained that it will display it.

"Operation case of Lychee Redmine and GitHub and a little interesting information"

Mr. Takashi Mizuguchi, Agileware Inc., Ltd.

Agileware is engaged in contracted system development, development and sales of the project management tool "Lychee Redmine", as well as development and sales of the real-time meeting minutes sharing service "GIJI". This time, he talked about in-house operation cases with Lychee Redmine and GitHub.

Lychee Redmine currently has more than 500 companies that have introduced it, and we are growing the product while promoting contract development according to customer requests and adding functions. Popular plug-ins include "Gantt Chart" for schedule management, "Ticket Board" (Kanban) for SCRUM, and "Time Management" and "Resource Management" for man-hour resource management.

Agileware has long used GitHub.com for code management during software development. According to Mr. Mizuguchi, the reason for this is that there is an optimal review process for engineers, such as quality control, an environment where they can concentrate on agile development, and engineer training.
However, there were some issues in linking Redmine and GitHub. One is that Redmine cannot work together without a clone of the Bare repository in the same server and must be done manually. Another is that change detection (push detection) is not standard. Therefore, in agileware, Redmine was limited as a visualization tool for management-side project management, task management, progress management, etc., and was not linked to repositories. On the other hand, it is said that GitHub was operated clearly as a communication tool between engineers for making pull requests and reviews.
As a result, on the management side, it was difficult to intuitively grasp the overall situation, and it was difficult to indicate the order of priority.

So the company developed Lychee Redmine. In addition to manually prioritizing with the Lychee ticket board of the Kanban function, you can also manage milestones of multiple projects in progress on the Gantt chart. By using it together with GitHub, it becomes possible for the management side to grasp the entire situation, and it also promotes the visualization of the progress. Changing priorities just got easier.

Manage multiple project milestones simultaneously with Lychee Gantt Chart

In addition, it is easier for engineers to understand the priority of tasks in Redmine, it is easier to update the status, and reviews can be performed on the GitHub UI.

However, Mr. Mizuguchi, who was still dissatisfied with the fact that it could not be linked in real time, revealed that he was developing a "GitHub linkage plug-in" using OSS. In addition to cloning repositories from GitHub on Redmine, we plan to incorporate functions such as a function to link with a GitHub account on the personal settings screen, and a function to create branches and generate pull requests from the ticket details screen. Mr. Mizuguchi emphasized that this plug-in will make the combined operation of GitHub and Lychee Redmine seamless, and that it will deepen cooperation between management and engineers.

"Macnica Private Docker Cloud"

Macnica Solutions Corp. Technology Department, 2nd Division Nohara Minehiko

At Macnica group, technical support for many products required an environment that included external software integration, analysis tools, and monitoring tools. Therefore, starting around 2015, we gradually migrated environments such as operation monitoring servers, web servers, proxy servers, and various database servers to Docker in order to alleviate resource constraints in virtual machines. At the same time as it became extremely convenient, we were also concerned about the lack of rack space and tight resources in each environment due to the proliferation of virtual environments across the company, so we began considering an application provisioning platform using Docker.

Nohara says that using Docker has many advantages, such as resource reduction, hardware environment consolidation, system portability improvement, high-speed deployment/scaling, and freedom from dependency problems. increase. On the other hand, he points out that there are many issues such as lack of user knowledge of Docker, compliance and security risks, monitoring and maintenance operations, and restrictions on stateful data.

Therefore, the requirements for introducing Docker are that no special knowledge is required, that operational efficiency is improved while satisfying self-service and internal security requirements. I chose CoreOS and Docker Registry for the registry.
However, not all the applications you want to use are in the Rancher catalog, and depending on the situation, you may need to customize or build your own application container, and the built application image will be published on the Docker Hub public repository according to the license agreement. It may also include things you can't do. Therefore, the private catalog configures the UI by including yaml files called Docker-compose and Rancher-compose. Each compose file is placed on GitHub Enterprise and can be implemented by specifying the URL of the repository from the UI. Docker images that can be published globally are automatically built using Docker Hub. On the other hand, Docker images that cannot be published are built with GitHub Enterprise and Jenkins, scanned for vulnerabilities with Twistlock (container security), and pushed to the internal Docker Registry.

Configuration overview of Macnica Private Docker Cloud

We emphasize that the Macnica Private Docker Cloud built in this way has reduced physical resources, reduced the man-hours required by engineers to build the environment, and shortened the time it takes to start using applications. In addition, it is possible to use it as a demo environment for products handled, to check the UI and operation of each application version, and to improve knowledge by sharing information with the OSS community.

Nohara said that in the future, he plans to scale the operation environment, migrate to the Kubernetes environment, and apply Twistlock to the Registry and the operation environment.

Finally, Tatsuya Nemoto of Macnica proposed total support in the form of close collaboration between GitHub Enterprise, CircleCI, and JFrog regarding the DevOps tools group provided by Macnica, and proposed that all sessions of the 2nd GitHub Enterprise User Conference be provided. has ended.

Macnica provides a single point of contact for inquiries to each vendor, and a dedicated team offers technical support in Japanese for installation, operation, and expansion," says Nemoto.

Even though the enthusiasm had not yet abated, many people attended the social gathering after the event, and there was a lively exchange of information with the speakers who were in charge of the sessions.