Security: Best Practices in Integrated Product Management with Peter Ganza

TRANSCRIPT OF WEBINAR:

I'd like to thank everybody who was able to make it to this WebEx presentation.

The reason we're here today is we've been spending a lot of here at Ryma Technology Solutions working with a variety of security companies in different fashions.

We've been learning a lot of interesting information from these companies. One of the things we wanted to do was be able to come back to the security market in general, and specifically product management, and give you a sense of some of the things we've been seeing in terms of some commonalities across these security companies and how they do their product management.

We're just going to go through some basic slides here and look at three themes in terms of some of the things we see as common problems. And of course we'll go into some of the common solutions that we've seen to those problems as well as a couple of specific examples around them; some reporting and some roadmap type of information that we've seen, to see across the board in most security companies.

We'll go ahead and jump in here in terms of a little brief agenda; just a little bit of an introduction here. I'll talk to you about who I am and in a moment, we'll jump right into it.

The three themes I talked about that we're concentrating on and that we're really have seem to found common ground across these security companies, are this third bullet that you see here: process, data and collaboration.

Those are the three key areas that we've found that there tends to be problems and disconnects in organizations today, and attempts to be a lot of different ways that you can solve or attempt to solve these problems. Those are really the areas we're going to focus on, again using some specific examples.

We'll also look at reporting and road maps a little bit just to talk about some of the similarities we've run across there in terms of our security customers and prospects. Then as a last slide, some information on helping customers and prospects build out their project management organization.

Product management is a very interesting space in companies. We tend to be the CEO of a product, for lack of a better term, because they have to work with everybody. Everybody almost has to give us something and in most cases we give them something back. We find it is a role that is moving forward in organizations. It's moving higher and higher and becoming a larger organization, and we've seen some common tools and things that are available around training and certifications in the market that we wanted to share with our customers and prospects.

The first thing to get out of the way is to tell you who I am and why I'm here.

My name is Peter Ganza and I am the Professional Services Lead here at Ryma Technology Solutions in charge of helping our customers get fulfilled with our solutions and actually working with them to build out our best practices for our solution, FeaturePlan, and tie that into their current lives and their product management process.

I've been in high technology for quite a while; just over ten years. Spent a good deal of time working at Symantec Corporation in a variety of roles around product management, marketing and some technical roles as well. That was one of the fun ways that I was able to build in a security kind of approach to some of these accounts that we deal with; to be able to work with security companies and really understand what it is they are talking about.

We all have the same kind of fun to deal with our own requirements, working with different groups - I found it was very useful for me in having that experience, in working with security companies now in this role.

I am Pragmatic Marketing trained, as we do support that methodology. And, again, really focused on professional services role in helping our customers.

What we are looking to do here is to start off; what are we seeing overall in product management in security. When I say security, I mean the whole gamut; Software security, hardware security, services or perhaps even a combination of both. We all love the new buzz words around integration and tying everything together, and it is the way that security is going.

We're finding that the same is true for product management within a security company. They are tending to become more integrated, just like their offerings. We need to do the same thing with our processes and collaboration that we are using internally.

Really, what we found is - this one picture speaks marvels not only to most of our customers but what we've found in the security space as well - where you are basically, like I said, the CEO of the product. Product management is here, over on the right, dealing with inputs from everybody.

You can see a couple of the common suspects here in terms of information coming from sales and development; of course our customers and suppliers are feeding us lots of information as well as every other group in the organization.

We find that in security, roles around product management, it tends to be the same process. The difference is, we find we're getting more data in security companies and it's coming in quicker.

We have more people screaming at us and we have people who scream at us more often than the traditional, say, software company. That's something we have found across the board. I think it's really speaks well to the fast pace of the security industry.

We have to move quicker than everybody else, vs, let's say, a standard software product. There's a lot more at stake. We're dealing with of course security, which is a big issue. In that sense, we need to move quicker than anybody else because of our customers.

If we look at a specific example here of what a product manager in a security organization would typically be dealing with, there are really two focuses; the things that are coming in and the things that need to go out.

It's pretty basic. I'm sure everyone will hopefully agree with most of what we're laying out here. At the top level, we've got a lot of different data coming in from different places. These are just five common examples where of course we potentially are tying into our CRM tools perhaps for win/loss data or tying into our sales process and moving that forward.

We're of course getting tons of enhancement requests and bugs coming in from the field and coming in from our customers. We have lots of field data coming in. This could be information from sales, engineers, support - people that are feeding us information on what's working, what's not. What problems people are having. What things they would like to see us develop.

We have a lot to deal with on the research side. I could have broken this out into analysts research or market research but I decided to leave it as one big bucket where we're dealing with all of the research.

One of the interesting things I've found, is for whatever reason in security, we tend to deal with research a lot more than a standard software company or even a standard hardware company for that matter.

We tend to live and die by our gardener magic quadrants and our IDC market share charts and our NPD data telling us what's selling through our distribution. We have found that is clearly a very valuable asset; something that we have to use.

As well we've of course got interaction with Development and potentially QA to push forward these things that are coming in from the field and build up requirements and do something with that; give us the estimate information, the specifications, the regulatory compliance issues - all of these kinds of things.

All of this data is coming into product management in one way or another. Your vanilla file folders, your voicemail, your email inbox. I don't think I've met a single product manager in security who wasn't working 60, 70 or 80 hours a week. That seems to be the norm. What these overstretched people tend to have to do, is take all these inputs that are coming and boil them out into something.

Those five boxes you see at the bottom are some fairly common examples of the key deliverables we tend to see out of security product managers.

Of course our MRD and PRD are huge. We're going to talk a little about those a little later. Those tend to be our number one deliverables. Depending on the organization you might be more leaning toward an MRD or perhaps towards a PRD and that's depending on how big the organization is and where some of these things may have ownership.

There's two huge ones, and we'll talk a little more about those. Business cases in the middle here are fairly common where we need to justify the business case behind something. It could be an entire release, it could be a service, it could be a feature. We need to justify something to someone.

Contracts are a great example --we're seeing a lot more of -- especially throughout these security organizations where we need to get people to sign off on something. Here's what we're planning to build next quarter. Here's what Development has agreed to; QA's agreed to - what everyone in the organization has agreed to - let's put that literally in a contract, get people to sign off on it, and then use that going forward so that we can understand who agreed to what and under what constraints or timelines.

Of course everyone's favorite at the bottom right there is our roadmap. We'll talk about roadmap examples at the end. This is of course a huge deliverable and we find it's very common for security PMs to be dealing a lot with roadmaps. These could be internal roadmaps for constituents in the organization to understand where we're going, and, everyone's favorite, external roadmaps where we start to be able to give our prospects or customers some information on potentially where we're going.

We will, again, look at a particular example of that and some of the pitfalls around it as well.

The place we really want to start, now that we have a good sense of the overall product management in security and some of the common things that we've seen, there's really those three areas we want to focus on.

Before we jump into that, you'll notice that at the end of the day, this is how we find our product managers aren't normally looking; very tired and quite overworked.

We talked about those three elements: process, data and collaboration. Let me talk about each of those briefly. These are the three things that I really was able to pull out of my experience in the last year of just working with these security companies - that these were the three biggest issues - but they were also the three biggest opportunities to improve things and build upon our process and the methodologies we're using, to save us time and build better requirements which in turn will lead us to better products.

The first part of that is the process. Everyone loves to talk about process; what's our process ... let's change the process ... let's build a new process ... it seems to be a buzzword floating around everywhere these days.

There are a couple common things we're seeing within the process around security companies. That first is the data and the outputs. We talked a little earlier about some of the data that's potentially coming in and some of the outputs - it's very clear to see that is putting a strain on our process.

We have to deal with more data; we have to deal with more outputs; we have to deal with more constituents and that puts a strain on the process. It's either a bottleneck or it becomes a barrier for us to actually achieve the process that we want to.

The reason we've seen that tends to be, is because of the disconnect between the methodology we're using and the actual processes that we're implementing. Everyone has a methodology and in most cases it's something similar to what you see at the bottom here where we're basically taking in some market inputs, defining it into something, some kind of a bucket like a problem, and then we're boiling that out into requirements which we actually go ahead and build.

This would be the most simple form of a high level, 30,000 foot view of a methodology. The methodology is just a methodology. In order for it to be successful, we need the process behind it driving forward the methodology, driving forward the milestones that were tied to the contracts that we've signed onto - or any part of the methodology.

It's something we need to define, and use it, to drive forward. There are some great examples of things we're seeing for solutions in process. It's something we're going to talk about in a moment.

So we've talked about the process issues, we're going to talk about the data issues and the collaboration, and then we'll talk about some of the solutions for those.

The data issues. Where is our data. We're finding this is becoming more and more of an issue; the amount of data that's coming in. It could be data that's coming in in reports, spreadsheets, or dumps of data; we've got emails going in, phone calls left and right, mental notes - I've seen some product managers with some very interesting spreadsheets to try and track this information.

I'm going to show you one quick example of one potential one and what the pitfalls actually were for that.

We're finding this is definitely a bottleneck. This is a place where PMs are able to get the data, but they're not able to get it in an easy way, in a consistent way or in a very manageable way.

We've got so many systems, data bases, tools and existing processes that are driving data towards us - just not in a consolidated or concentric way. It's coming in a variety of different formats; a variety of different times and the problem we are seeing is it's hard to get a handle on all of the data.

We need to have it all in one place. We need to be able to look at it, analyze it, make decisions on it, and get away from basically the fire hose approach, where we find most security product management organizations are today - you're putting out fires. The requests are coming in and we're putting them out as we can, and really not being pro-active. It's more of a reactive approach.

So we have some data issues we need to resolve and we'll talk a little bit in a moment about some of the potential solutions we've seen to those problems.

Last but not least the third item we want to focus on here is collaboration. Everyone can agree on how important collaboration is; everyone is doing some collaboration. You have to. As I pointed out in the first bullet here, stakeholders that we're dealing with are growing. It's true for all product management organizations as well as security ones where we've got a boat load of this two way communication that has to happen.

We're finding that it's difficult for people to facilitate not only that collaboration or that communication but the amount that it's growing. There are so many other constituents we're finding that are being pulled in - groups like business development that we're working a lot more closely with now than we used to - say five years ago - people in marketing or other constituents in the organization that we didn't perhaps work with all the time in the past or in a very process-driven way but today we're getting to the point where we need to have some more process.

We need to have a way of collaborating with these people that's easy. The phone calls and emails and all of that's great but it's very difficult to manage. Did I talk to so-and-so, did they sign off on that milestone, did I get back to someone in business development around a viable partner decision.

Again, it's a management issue where we need an easy way to facilitate and empower people to collaborate. We want people to collaborate. We want t heir information, we want them to help us with our parts of product management, and there needs to be an easy way to do that.

We talked about the three keys there; process, data and collaboration, and I've started to paint a nasty picture here in terms of the problems that we are seeing common across security companies.

We want to talk about some of the solutions that we're seeing, and ways that people can help in these three disciplines.

The first thing we want to talk about is process and some of the solutions we're seeing here.

First and foremost, I can't speak highly enough for having a well defined methodology. It could be a methodology that's out in the industry today. I put four examples down at the bottom - things like Pragmatic Marketing and a few others - where there are methodologies out there today where we can get trained on, we can perhaps even get the consulting on to get a best fit into our organization, and we can drive forward some type of a methodology to move us to where we want to be, which tends to be market driven.

We're finding a lot of organizations, especially in security, have to be customer-driven. We're just starting out ... we're in a very interesting space ... perhaps a niche within a segment of security ... and we really have to build our products and services based off of our customer information.

People are starting to realize that we need to pull in more market data. What are not only our customers saying but our prospects. What is the market saying. What are analysts and the gardeners and the ITCs actually telling us.

We need to have a methodology for us to understand what data is valuable to us. Are we going to be customer-driven or market-driven or development-driven. Let's pick one methodology and let's drive it forward.

A lot of things will boil out of that, of course; organizational roles and responsibilities are huge. I can't tell you how many times I've been on the phone with some of our customers in security where roles and responsibilities tends to be our barrier. Who owns what. Who is responsible for what.

It really amazes me in this day and age that we still have so many barriers around; who owns what and who does what.

One of the things we've found is companies who do have methodology, like let's say Pragmatic, for instance, who do have process behind it that they've built in and that they are of course using and modifying as time goes, they don't have as much or any of these organizational issues - or at least not even close to an organization who doesn't have any methodology or process would potentially be at.

The custom consulting piece is an interesting one because one of the key things we've been able to pull out of working with these security companies, is that the methodologies are great but they are at a very, very high level.

If we take Pragmatic Marketing's course, for example, we learn all this great information about how to be market-driven at a 30,000 foot level but when we come back to work we realize we can only implement maybe 30 or 40% of it because of the way the company is built internally.

That might not change and perhaps it doesn't need to change. But we need to use process to drive forward the pieces of the methodology that are valuable to us and that are going to be useful to us.

These are a few examples of some of the companies that are out there offering different methodologies. I won't even list consulting firms because as you can imagine, there's a boat load of those out there. Everyone has a few phone numbers for consultants, I'm sure.

So it's a process piece really tied specifically to methodology to drive the two together.

On the data piece, we're seeing an enormous amount of tools out there today. It tends to be broken up into three or four segments that you see here. The first one is Development and QA where we're definitely seeing in almost every organization, that there is a tool and usually some process that's already in place in the organization.

There are four examples here of some development tools we tend to find fairly common amongst security organizations - probably Rational's suite of products as well as the Boreland Whitby on the top layer, as well as Mercury and MKS which are two other companies offerings which are great for development.

They are tools that Development can use to manage their requirements; what piece of the code they are modifying; what DLL they are checking in or checking out and all those low-level, in the weeds specifics on who's building what and what are they actually working on.

The next layer we tend to see is the CRM tools. We all have one whether it's an Oracle, a Siebel, a Salesforce - maybe perhaps something from Pivotal - or others - and this is generally something that's not being driven by product management but we need to have this as part of our sales tools for sales people to manage all their information.

But as a product manager, there's a wonderful piece of data that oftentimes we can pull out of these tools, which is win/loss data. Everyone love's win/loss information and a lot of people talk about it. We find that it's true value is if you can get both and they are of course useful.

Let me expand on that a little bit.

The first thing that we found is that people don't tend to get both. They either take just wins or just losses. We find that's a detriment to the process. We need to understand not only why we're winning but why we're losing as well as the other way around. We don't want to look at just one side of the coin, we want to see the full picture; reasons that we're winning and reasons that we're losing because they could be tied together or they could be similar or they could be on two totally different ends of the spectrum but wherever they happen to lie, it generally points us in a direction that we can clearly see trends and matrix for why we win. Is it because we're the fastest, the cheapest, the quickest - and reasons why we're potentially losing - maybe it's because we're too expensive or because we're too slow.

We can generally use both of those points; wins and losses to end up in the middle somewhere and make some very clear decisions on - we need to be more performance-based or we need to work on our licensing and pricing or our terms and conditions. Whatever happens to come out of those. But we need both.

The other thing we find missing is the drive and the champion internally. Sales needs to be empowered to give us this data in a useable way. When I say useable way, we need to know why we're winning or why we're losing. We don't need to know that you lost this deal because of account management or because of price. We need the details behind it. It can't be a checkbox or a pull-down where the Sales people pick the same thing over and over again.

After CRM, we move into a more customer-facing role where we all end up with lots of tools for bugs and enhancements - Bugzilla is probably the most common we see for tool tracking, bug reports and enhancement requests - the Office tools and Sharepoint is probably the most popular. We might have a portal built or some way of having a web data base to have some way to pull in these kinds of data.

These are all great tools for what they do specifically. You can see they are very much focused on the areas that they are dealing with. A Dev tool is great for Dev; a CRM tool is great for Sales and Support and a bug tool is great for bugs. But that's about it. It's very typical to cross those barriers.

That covers some of the solutions we see around data.

Around collaboration, we're not seeing enough today. There's two approaches; in automated or manual approach. Our automated approach is basically - we're trying to use Office - perhaps in conjunction with Sharepoint to maybe project, to manage our tasks or our interactions in some way.

We might have a data base like a Lotus ___ database where we actually do some collaboration in a newsgroup thing ... so we're starting to see some tools developed around that. A lot of them tend to be home grown. Some of them do get the job done. It's definitely an area where we're still missing something. We're still missing a good way to instill the collaboration, store it all and be able to use it in some valuable way.

Those are the three areas; the process, data and collaboration and some of the solutions we're seeing around those.

Next thing we want to look at are some specific examples that came out of those. The first is reporting. I mentioned earlier that our number one deliverable we see today for security PMs are MRDs and PRDs.

The MRD tends to be more higher level information whereas the PRD tends to be much, much more in the weeds. So the difference in size is fairly great. If your MRD is 10 pages, your PRD is probably 20 or 30 pages.

One of the things that was clear to us in looking at what our customers are doing and using today around security, are the weights and the measures.

I'm going to give you two specific little examples that we can talk through which I found very interesting. The first is what we are using for weights and measures, and the second is how much we are using.

Let me just put the slide here and describe to you these interesting screen shots. The first thing you are looking at here is just a quick example of some of the common things we've seen our customers tracking in their spreadsheets for their particular requirements for each requirement.

For each requirement, they would be listed out as rows here, and they've got a ridiculous amount of columns. This is an example. I've seen some worse ones. But we have a big set of data for each particular requirement. What's the need. What's the impact. What's our strategic value. Our priority as well as a whole bunch of other things here around risks, revenue, complexity, staffing.

Basically people are tending to build spreadsheets like this, put in magical values which I still wonder what crystal ball they come out of, and you do something with those numbers. We might add them up and give totals like you see here. Of course, these totals are going to tell me which requirements are more important than the others by looking at the different variables.

The problem that we see in t his is the amount of these variables and the amount of data. That leads to the second screen shot here which hopefully some of you will recognize as the page count in a Microsoft Word document. .This is basically me opening up an MRD example from one of our customers, and looking at the sample they sent me which was 381 pages.

The first thing we told them, was that's not a useable document. Anybody in the methodology or process perspective will probably tell you any report that's over, say, 75 pages no longer becomes useful.

A 381 page MRD is almost useless. Nobody is going to read 381 pages. People are going to go in and pick and choose things based on the table of contents; they might do a search, or in most cases they're going to pick up the phone and ask you anyway, even if the answer is in the MRD. They don't want to go through 300+ pages looking for it.

We need to whittle this information down to just what we are using. All these variables we are seeing on this list - staff, complexity, revenue, risk - these are all valuable variables that at some point or other we may want to use and want to track. But we don't necessarily need all these things in our plans, in our MRDs and our PRDs. We need just enough information.

What's just enough information? That's the question. If you look at this next slide, just as a quick example, some of those variables, and what a mess they actually tend to be - the key here is the prioritization.

We need to prioritize the information that we need to put into these documents for people to make their decisions. We probably don't need 12 variables for someone to look at and say I want to do this requirement or I don't want to do this requirement. We really only need 3 or 4.

The three common ones that we've found, are these that you see here - quantifiable numbers that we can understand: value, priority and impact. Value is going to be a generic; what is going to be the value of this to us. If we put in this feature functionality or we meet this regulatory compliance or this performance barrier on the network, we're going to close a million dollars in business. That's quantifiable; there's a number behind that.

Priority. What's the priority. Is this something that ties into our strategy or is this an internal goal that we're trying to do anyway. That's huge and that's quantifiable. Anybody can relate to a priority as long as it takes into account all the factors.

Third would be the impact. What effect is this going to have on us and potentially on the market. Do we have a competitive displacement opportunity. If we build this feature, are we going to kick a competitor out of a big account. Are we going to be talked about in the market as being visionary. Are we going to leap ahead the competition that's out there today.

There need to be quantifiable things, and we don't need 12 of them. We were finding when we talked to these product managers, they just got too much information. It's all valuable information but for certain audiences. We don't need to give everybody all the information.

We need to pick and choose who we give what, and just give them what they need to make the decisions or to give you whatever information they were planning to give you.

The next thing I want to take a look at here after the reporting examples, are just a little bit on roadmaps. Roadmaps are definitely a hot topic. We talked about it before; there's always this discussion about what is a roadmap; is it internal, is it external - and basically everyone's told us it's a problem.

How do I build a living, breathing roadmap, an internal version, an external version, what's the grey area between t hose - and how can I make them interactive in a way I can drill down into the information.

I could sit here and tell you about how we can do it with our tool but that's not what we're here for. What are people in security and product management doing today.

There's a couple quick examples I've built out here that should give us a real brief understanding.

The first thing we want to do is talk about dates. We don't want to put dates on a roadmap. Because of course we're now committing to something. We're committing to a future and we're basically doing what our Sales people often do too much of; promising something that we may not be able to deliver.

Pragmatic Marketing has a great idea where they say your dates should actually be 'no sooner than' dates. So if I say I'm shipping March 1st, you're not going to get it before March 1st.

You may not necessarily get it on March 1st. It might be March 32nd which tends to happen a little too often - but a 'no sooner than' date is something we're starting to see as an interesting thing.

In line with that, is building these roadmaps that are for periods of time. In this particular example I've built a basic roadmap as you can see, where on the left I'm defining six periods here; the first four quarters, the next two and half years. This is very high level. I'm not committing to a specific date. I'm not saying in Q-1 of 2006 I'm going to release imaging features and definition feature into my particular product. That could be for whatever product I'm looking at this roadmap for.

I've also got some associate information to give me some data there on what market this is addressing; perhaps if it's tied to a goal or a strategy - and then some low level information for is it part of the interface, what part of the architecture or what part of the platform it may use.

This kind of format is something we're seeing people trying to get to because it is something we can reproduce. It's something I can cut and paste, and send to 10 different people in 10 different ways, if that's what they want.

For my customers or prospects I can take out the last five or six columns. They don't need to know that low level information. They might just need to have the high level. When, what - potentially the how. Maybe for some people internally I am going to leave all this information.

Maybe development needs to understand what part of the architecture or the platform we're dealing with in a particular course of time or particular period.

Let's just look at example of a high level; what we tend to see people trying to get to in relating this information. It's great because if we can build one baseline, we can basically cut and paste that and build it into additional slices for the different people but still managing everything within one place.

If we drill down a little bit further, we can get to something that you see like here - where we are actually taking t hose particular periods of time and breaking them out into a lot more detail. This is something I've seen more and more common especially internally where we need to give these kinds of roadmaps to executives or someone else in the organization with a little more detail.

So we break out a period of time here - Key 106 - the two features I'm planning to release - and some basic information that happens to be behind them.

The keys I found on the roadmaps are going to be the dates. Let's not give dates anymore, people. Let's give periods of time. We find we can stick to those generally a lot better than a date.

If we do have to give a date, tell them it's a 'no sooner than' date like Pragmatic would tell you. You can't expect it any sooner than this date. You're not promising to that date but it's a 'no sooner than' date.

The second thing is having our baseline roadmap that we can build from. It's always easier to have one big one that we slice and dice as we need to, than managing five separate ones.

That's common and we've seen that across the board.

The last thing we want to touch on here are some examples and tools - some information that's available for building out a world-class product management organization. There are options out there. There is training out there. There are sets of methodologies out there. There are a lot of consultants that can offer a lot of different things.

I'll just take you through a quick list here of some of the things we're seeing security companies use to build out their organizations for product management.

The first are the training options. Of course, I can't talk about Pragmatic Marketing enough. They are the biggest and most well known product management methodology in the world. They've trained something like 35,000 people. There are others. ZigZagus is one of the ones out there - there's one called the PM University.

I can't talk about the product management associations enough. If you are in any big city anywhere in the world, you probably have a product management association in your city. Just for example there's a Toronto Association, a Boston Association - and a Silicon Valley Product management Association. I cannot speak about these any higher. They are a phenomenal opportunity to meet product managers in a variety of different roles; in a variety of different organizations and verticals and segments; to network with people and learn about the ways they do things; their challenges, their solutions.

They've always got some great presentations on items that product managers care to tell about. I can't speak about them enough. They cost next to nothing. It's usually about $10 to get into these meetings. They usually do them in the evenings. Like I said, I cannot speak highly enough for using those to help you learn more about product management and of course find product managers.

There are some strategy networks out there like the Pittsburgh Strategy Network is one - that provide some training and some templates. If you need to build out MRDs or PRDs or you want to have a more in depth understanding of how to build roadmaps, there's three or four out there--I'd be happy to provide you some more names - the Pittsburgh Strategy Network is the first one I think of.

Outside training options, we do have some hiring options on where to go. Associations, again, are huge. I found this job through the Toronto Product Management Association.

I can tell you in almost every meeting, there are always a couple product managers who are looking for work. It tends to be a great place to find talent. It could be a junior or senior PM, it could be even a director or a VP level as well.

The other thing we're seeing, is a lot more product management people are getting on line into newsgroups and blogs around product management, around marketing.

We've got a blog on our site where we're building up some product management expertise. There are a few others out there. We're starting to see more and more of it, and more people actually getting into it.

It is a great resource to meet with some people and potentially even find some help.

Last but not least, when it comes to recruiters, it was very clear to me in just talking to four or five customers - they all said the same thing. You can get a basic recruiter or a recruiter that understands product management. The basic recruiter is like everybody else; they're going to look at the line items that you wrote down and try to find you someone who matches that.

Whereas the recruiter with the product management expertise is going to be able to look a little bit further. They are going to understand what you're looking for; are you looking for a technical PM or a marketing PM. Does this person have to have a technical background or more of a strategic background.

It was very clear to me in the first few calls, that there's a big difference between recruiters who have that experience, and don't.

So if you are looking for product managers or any component of your product management organization, if you're working with HR or working with recruiters, just ask them a simple question. Have you worked with product managers before or have you gotten a product manager a job. That tends to be a good place to start to understand 'does he really know what I'm talking about' or are they just another recruiter trying to get their cut of the commission or whatever it happens to be.

Just a few options that are out there.

Last thing I'm going to close with here, is a little bit of a wrap-up.

The three keys that we found in working with product management organizations who are working with security, are the process, the data and the collaboration. I can't speak about those highly enough because again, those are the three biggest problems we see - but also the three biggest opportunities.

If we line up our process with their methodology, if we line up our data in some way, and we instill collaboration, we're already halfway there just by lining up those three big buckets. We start to find organizations can move quicker; product managers can have less on their plate - I know it's hard to believe but it is true.

We can do things like automate reports and not have to deal with spending a third of our time every week writing up MRDs or PRDs. Or any piece of that tactical puzzle; a lot of these have to do with tactical things - almost administrative things that we waste our time on. Any way we can find to automate these parts of our process or parts of our data collection or our collaboration are going to be enormous for us.

They're going to save us a ton of time, generate some ROI for us, and give us time to work on more strategic things; not worry about enhancement requests but worry about which competitor to kill or the next move Microsoft's going to make to run us out of business - so spend more time on these important strategic things instead of these low level tactical things.

The last thing I am going to show you is the slide with our website and our email address. I would definitely like feedback.

Have a great day. Thanks for attending and we'll talk to you soon.