Putting Product-Building Practices Into Action

Products matter, which means how you build and support products matter. In his presentation for “Product Matters,” Alex Tomșa walked attendees through a real software product from 3Pillar Global to showcase how our development teams put our product-building practices into action. Product Matters is a product development-focused event that was held on September 7^th in Iași, Romania as part of the opening of 3Pillar’s new office there.

Alex’s presentation, titled “Showcase: Cloud-Based Platform for Integrated Services,” centers on a search firm management tool and online recruitment marketplace where he and his development teams are applying scrum and agile methodologies to delight the customer. He also highlights how the development for this product was unique, focusing on the specifics of scrum mastering multiple teams spread across different time zones but still working on the same backlog.

About Alex Tomșa

Alex is an Engineering Manager at 3Pillar Global. Being an engineer at his core, he followed a path of systems design, implementation, and tuning throughout his entire career. From manufacturing processes automation to quality management for the automotive industry, he always looked through his engineering glasses for harmony in processes. Software industry provided him with the best place to apply the engineering mindset at all levels: technical, product and management. He is excited and thrilled about a world where boundaries are set only by our imagination.

About Product Matters

“Product Matters” was the first 3Pillar tech event in Iasi, Romania. The conference featured presentations on recent technology trends and understanding how teams work together to better manage a product’s lifecycle. Attendees were also taught the Product Mindset during the conference.

Read the Transcript

We talked a little bit about theory and then some practice from Oana, so I’m going to try to present you how we do things with an actual customer. We are talking about the fact that product matters, and just want to show you how we apply everything that David, Bogdan, Jonathan, and Oana said so for to an actual, real life product that is happening right now in 3Pillar. As Jonathan said, this is one of the 100th, 150 products, so it’s just an example.

I will go quickly through the client, the product, tech and so on, then if you have any questions at the end, I will build my presentation upon your questions.

First of all, about the product. This is what the client actually says about itself in his own web page. The world’s largest search firm management tool and online recruitment marketplace that delivers results. What they actually do, they connect companies with search firms and third-party recruiters just to fill in jobs. My definition would be, shortly, it’s an Amazon for recruitment.

The product itself, it’s quite large, so there is a marketplace platform. This is where all the features are gathered together, and they are presented to the customers. Then there are a lot of integrations, ATS. ATS means applicant tracking systems. These are the systems that are inside companies that are managing the HR departments, let’s say, more or less. All these systems are connecting into the marketplace from Scout, and they are communicating with each other.

They have an interesting AI for resume matching. This means they are building upon a system which actually is continuously searching to match resumes with job postings, so that they find the best match between them. They have, of course, a lot of supporting services, authentication systems, elasticsearch, data warehousing, reporting tools, everything that a usual product needs.

Then, they have a separate system which does time sheets, because they also handle temporary jobs, and those temporary jobs fill in their timesheets directly into this cloud product, and then they generate the invoices for them.

The technology that is built upon, it’s – for Front End, we have React JS with Backbone JS, HTML5 and SAS as a CSS preprocessor. It’s built with Grunt and Web Pack, and the packet manager: NPM, which everybody knows.

On the back-end part, we have Python, so this is the main language used with Flask. Databases are MongoDB, MySQL, there is also Postgres in some parts. Also, they’re using elasticsearch from Amazon and Njinx. For continuous integration now is Jenkins, Vagrant, Puppet, and lately, they are starting to switch to Docker.

What Jonathan said about the latest trends, and also there is a higher QA automation component, which is using Selenium WebDriver Behave, and then there is a framework that is built between them, by us.

The teams that are building this product, there are five feature teams. They are just handling features. They are just building features, nothing else, but as any product that is growing, and as you all know, supporting the product takes much more work than actually building it.

This supporting means there are five feature teams and six teams which are handling support, QA Automation team, DevOps team, Data Warehousing team, they have invoicing and authorization team. Also, they are doing research and development, there is a full team who only does research and development, and of course, there is the UX support team. In total, there are 48 engineers which are working for this product.

What is the role of 3Pillar within the product? 3Pillar started the relationship with Scout around 2015. This is about two years after Scout was incorporated as a company. Right now, 3Pillar has one fully independent team, a feature team, amongst the five that I just told you about. We kickstarted the QA automation process, and we started to build a framework that it’s handling automation for Scout.

There is one scrum master, myself, which is doing scrum mastering for the whole teams together with one assigned person from Scout. We share responsibilities equally. There is no difference between 3Pillar and Scout, which is actually something that is happening all around the project. The 3Pillar team is treated equally with Scout’s team, so there is no difference of any kind between us and them. This means everything that you can imagine, emails, group emails, meetings, important meetings, staff meetings.

Scout week is twice a year. This is happening in Boston. All the teams which are working remotely are called in Boston to meet. Amongst the 48 engineers that Scout has, about 20 are remote workers around the world, so they are spread across US and also in Europe – Spain, Holland, Romania, but we are all part of this big team, 48. Also, everything we do, it’s equally treated as their own team, so our contributing to the Scout architecture equally with them. We are tuning the process, the development process, together with them, and we are also mention coordination within and outside Scout, so we are managing actually contacting their third-party suppliers, their integrators, the same level that they are doing it.

This just happened, what I added here. Scout is a promoter for 3Pillar group. What this means is that Scout actually recommended us to another company, which already had some third-party suppliers, which they were not very happy with, and they said, “Look, we are working with 3Pillar. They’re great. Why don’t you want to give it a shot?” I think it took about one meeting to set up the new customer for us, and we are building a team as we speak for them.

A little bit about the process that they are using to build the product, because we are talking about product. These are some of the challenges that this kind of product has. It’s complex, first of all, so it has a lot of modules. These modules are not independent. They interact with each other. The feature teams that are assigned to build these modules are working independently, but actually on the same backlog.

The teams are spread geographically, as I told you. Sometimes there is only two or three, or maybe four hours overlap between time zones, and you have to manage all the meetings within this time zone difference.

There are a lot of external dependencies with third-party suppliers. The ATS, they are complicated, so we connect to their ATS using all the technologies we can think of, including FTP to gather data and push data to them, so it’s not everything restful.

Yes, the accelerated release schedule, so two years ago, I think, they had two times a year releases. Now they have monthly releases, and they want to push for a continuous release, so this means when the feature is done, it’s going to production automatically.

The organization is growing. It’s actually reaching the phase where Scout is not anymore a startup. It is going to become, soon, a full-fledged company, because the customers they have are growing, the customer base. They have Bank of America, Whirlpool, Red Bull. These are just the names that we in Romania know of, plus a lot of companies in US like Sentine and some other really, really big names in US.

How is it implemented, the approach? It’s typical approach. There are scrum teams. Those 11 scrum teams, they use Camba also, but they switch to scrum because it’s easier to manage, centralized. Two weeks sprints there is a scrum of scrums two times a week, where all the team leads from all the teams are gathering just to exchange information and updates about what they are doing. As I said, we are two scrum masters handling all this work for them. Not work, but managing the communication between the teams, and everything is built around supporting these monthly releases for everybody.

How this system is implemented. In terms of product, this is a very important thing – it’s useful for everybody to see how it works. They have the product managers embedded into the scrum teams, into the feature teams. This means that a product manager is responsible for other teams delivering. They explain what they need, they participate in all the ceremonies, the scrum ceremonies, all the stand-ups or the grooming planning as long as the feature is developed for them, and they are responsible also for the delivery of these features and what the teams are doing.

The product was developed using this term, architecture by committee. This means that there is no assigned architect for Scout for the entire product. Every time there is a big change which is impacting the product or impacting several teams, or may cause disruptions, the technical leads are gathering, they are deciding which is the best solution at that moment in time, and within the parameters given by the problem, and they decide for the approach. This, of course, has advantages, disadvantages, but so far, it’s working great for Scout. The product is growing. It’s sustainable, so this is a good approach for now.

The scrum master role. I will be talking a little bit about the scrum master role, but it has a key role to make everything smooth for the entire product team. Also, what is very, very important is that the entire organization structure is built to support the product. It’s not just there. We have the business, and then we have the product. No, they are working together. They are communicating a lot, and they are actually changing if the product needs to.

The scrum master role. I wanted to talk a little bit in more details about the scrum master, because I often get questions, or I see around what other teams are viewing the scrum master. I heard a lot that the scrum master is shifting inside the team. The role is moving between members from meeting to meeting. It’s coupled with the project manager or engineering manager for the team. Sometimes the scrum master actually removes roadblocks for some of the teams that I worked with or I’ve seen.

In some organizations, they have someone which is trying to be a scrum master, and then this person is organizing the entire development teams to work in an agile format, but he’s not all the time into the meetings. He’s just there to organize the organization.

Let’s see how Scout is doing it. This happened before I joined Scout, so this is something that they were already doing. The scrum team that the scrum master in Scout is managing is actually the entire product team. This is the scrum master’s team. It’s not just one of these 11 teams. I heard in one presentation in Boston someone saying that a good scrum master is mastering two teams. A great one is mastering one team. I was thinking to myself, okay, we have 11 teams. What are we doing here? It’s not 11 teams. It’s just one. It’s the product team. We split it because it’s easier to have overview of what’s going on, but there is only one team, actually. What we are splitting actually is the effort. It’s not the scrum master role. It’s just the effort.

The scrum master has actually access to all the C-level meetings. C means chief, chief executive meetings. He knows the release planning. He has complete oversight of what the product wants to ship and when, and he has something to say about it. If something feels wrong, we get to ask “Are you sure?”. Very deep product knowledge. You cannot scrum master a product if you don’t know it. That’s a given. Also, it helps a lot to have leadership expertise, not just scrum mastering expertise. You will see why, at this level, because these are some of the challenges that I faced, especially here, not in my entire career, but here it’s a very good example.

You get to ask very tough questions, very, very tough. Questions, that are impacting maybe millions of dollars’ accounts, like are you going to ship, really? You really think so? You don’t get to bring down people. You have to keep morale up. You have to keep them motivated while asking these tough questions.

Also, you have to keep the release schedule. You have to help keeping it, but without pushing the teams. They must not feel pressure, because they are developers, they are in the clouds, everything is perfect for them, and if you push them, they will just leave. You get to guide them, but you don’t get to intervene in what they are doing. They have to take the decisions for themselves, because they are the ones responsible for delivery. We are not, the scrum masters. We are just there to make sure that they will deliver, but we are not responsible. We get to guide, but we are not allowed to intervene. These are just three points. Maybe there are a lot more that I could talk about, but I don’t have time.

Now I want to give you a few examples. One of the most important ceremonies in the agile approach is the retrospective, and I saw many teams doing it just for the sake of having the retrospective because this is the agile format, and they have to keep it, and they did it, everybody got a laugh, or they had some fun, or they actually talked about their problems during the sprint, and then maybe they took notes, maybe they didn’t, and the notes got left over there. Maybe someone looked over them.

Here, it’s not like this. We do retrospective for all the teams, actually. Eleven teams. These retrospectives are actually reviewed once a month by the CTO of the company, and some examples that- actually, I think we don’t have enough time for examples, but if we see a pattern happening for the teams, this pattern gets addressed. One good example is the environments are not working. Five teams are complaining that okay, we spent one or two hours setting up environments. If you had only one team, okay, two hours, we got to finish the sprint in time. Everything went well. We don’t need to address this one. But, if five teams are telling you okay, we need two hours to set up the environments, this is a system problem. It’s not just the team’s problem. This is something if you address, it will improve velocity, team’s morale, and they will actually see that you care as an organization of their work.

Then, why it’s very important to scrum an entire product. Because you have oversight. You know what’s happening at all levels, including chief executive levels. This means you get to ask even tougher questions for everybody.

Why are we here? Because product matters. You cannot do this, if you don’t have a mindset, a product mindset, as a technology company. You cannot understand what the client wants from you if you don’t think in product language. We are not just technology guys. We are technology people, or technology companies, which are building something. We have a purpose. We are doing something, not just to be there. It’s exactly like this. Technology with a purpose.