Episode 189: What Happened to HackerOne with Joel Margolis

Title: Transcript - Thu, 27 Aug 2026 13:55:38 GMT
Date: Thu, 27 Aug 2026 13:55:38 GMT, Duration: [01:14:19.85]
[00:00:00.86] - Joel Margolis
These are the people who are using the platform the most. They are doing the most legwork. They are interfacing with stuff on the platform the most. And when they have some feedback, that we should be doing something with that.

[00:00:14.11] - Justin Gardner
Get it done. 

[00:00:32.67] - Justin Gardner
All right, real quick, we got the This Week in Bug Bounty segment. This week we've got 2 items. The first one is Yes We Hack has released a Claude Code Kit that they've developed that helps people understand whether their reports are ready to be submitted or not. And I think this is really good. A lot of people have been asking me what my bug bounty report writing skill looks like. Uh, it's essentially over and over and over again. Make it simpler, make it easier, make it simple, make it easier. But, um, Yes We Hack has, uh, done a really good job here of giving you a nice launchpad if you want to create one, or if you want to just use theirs as well. Um, we'll go ahead and link that into the description. It's something you can install from the plugin store, and, uh, it'll walk you through the process of giving a high-quality report to Yes We Hack. So great stuff here. Shout out to the Yes We Hack team for getting this shipped and out there to the community. All right. The second item on the list is an interesting one. So this episode is with Joel, uh, and he has some choice words about HackerOne. And, uh, you guys have probably seen his blog post that came out a while back. Um, and we sent the episode over to HackerOne in advance of releasing it, and they gave us, uh, a response from the CEO of HackerOne. So I'll go ahead and read that now. Uh, Joel, Justin, I read Joel's post and listened to the episode myself. You raised many good points. The part I want to fix first is how we exchange inaction feedback from the community. I don't have the full fix yet, but I own it and I'm also open to working together to find a good solution. Kara Sprague, CEO, HackerOne. So that's her response. Uh, now I guess you'll have to go listen to the episode and see what you think. Um, I think it's a good one and it was great to have Joel back. All right, let's check it out. You know what, Joel? I think this might be a standing episode.

[00:02:17.02] - Joel Margolis
Oh boy. Has it really been 5 years?

[00:02:19.18] - Justin Gardner
Dude, it's, it's been almost 5 years. Isn't that crazy? Oh my God. Gosh, dude, welcome back to the pod. I am so amped right now for the return of Joel. Um, obviously, maybe obviously, but for most people listening, uh, Joel wrote a blog post, uh, recently that has shook the bug bounty world to its core. Um, and as a former host of the Critical Thinking Podcast and co-founder, We must give him the platform to do it real time. Um, but before we jump into all that, um, welcome back, Joel. Dude, what have you been up to, man, for this past 2 years since you've been off the pod?

[00:03:00.33] - Joel Margolis
I don't know. I've been busy. I've been busy. I'm working at, uh, I switched companies. Um, I'm working at, um, at a crypto company now and, um, you know, I'm doing, I'm doing some bug bounty stuff over there and yeah, I've just been grinding. Um, getting ready to have my first kid and just Dude,

[00:03:16.62] - Justin Gardner
Dude, so excited. Like, like very soon, right?

[00:03:18.38] - Joel Margolis
Very soon. Couple days. Couple days. 

[00:03:20.71] - Justin Gardner
Dude, very exciting, man.

[00:03:22.11] - Joel Margolis
So yeah, yeah. I mean, life's been good. Life's been busy and just, you know, I've been watching from the sidelines cheering you guys on.

[00:03:29.52] - Justin Gardner
Thanks, man. Well, I appreciate that. And, you know, we definitely have planned on having you on as a guest more often than we have, but, you know, it is what it is. And yeah, dude, I think your blog post hit really hard, man. You know, it was a hard read. A lot of those points landed really, you know, close to the heart. Right. Um, and yeah, let's, let's go into it. Also, I just want to say to the listeners before we jump into this, thank you. Because I remember, like, as I power up this, you know, Riverside software with Joel here, I remember the beginning when Joel was here every week and you guys, you know, when Joel left, you were like, I don't know if we can deal with Justin's shit every week, you know, without Joel there to like— Call him out on it. But you guys have, so thank you. Dude, what was the impetus for this blog post? What the heck is going on and why are you taking shots at H1, man?

[00:04:25.54] - Joel Margolis
Okay. All right. Yeah. So what I'll start out with is a lot of this comes from private conversations that I've had with friends and other hackers in the community. A lot of private chats have centered around this. a lot more candid than this blog. But, you know—

[00:04:47.83] - Justin Gardner
Oh, shit. More candid than the blog?

[00:04:51.70] - Joel Margolis
Yes. Yes, I will say the blog is highly edited. So, you know, you should have seen v0. It was, it was not good. But yeah, you know, I think these are a collection of thoughts that I've had for a long time, and some of them have made their way out of the private circles into less private circles. If anybody's in the HSM program, You might've seen I made a big post a couple months ago in there that was very similar to a lot of the stuff that I talked about in this, a much more sort of condensed, more emotional version of this blog. But yeah, I mean, I think for me it's extremely high contrast when I, when I look at what HackerOne is today versus what HackerOne was when I first got involved. I first started hacking on HackerOne in mid-2017, right before H1702. Oddly enough, the last blog post that I posted almost 10 years ago was H1702 CTF write-up.

[00:05:52.81] - Justin Gardner
No way, dude, that's hilarious.

[00:05:55.27] - Joel Margolis
It's been a bit of a— it's been a bit of a blog hiatus. But yeah, I mean, you know, at that time HackerOne was such like a grassroots company is almost how I feel about it.

[00:06:10.06] - Justin Gardner
Yeah.

[00:06:10.39] - Joel Margolis
They were super connected to the community and they were doing a ton of work to sort of bring hackers up out of the shadows and create an awesome community and create a safe space and create these amazing sort of environments where hackers could connect and they could create huge amounts of impact for companies and do all these amazing things at live hacking events. And I think like you know, HackerOne was doing things before live hacking events, but for me, like live hacking events were a big turning point for HackerOne. I think, you know, people were doing ethical research and all that kind of stuff, but it was much more niche and quiet and not as talked about. And when they started really leaning into the live hacking events, that was like an amazing sort of both publicity for them, but also it created these, these networking and community-centric, you know, points where they could fly people in and people could just talk and you can meet these people that you only knew their handle or you'd maybe seen their research or stuff like that. And, you know, it was a really, really amazing time for HackerOne.

[00:07:18.69] - Justin Gardner
Yeah.

[00:07:19.43] - Joel Margolis
And that lasted for quite some time. You know, I think that probably up until COVID, was sort of, you know, like I talk about in my blog, that was kind of like the golden era for HackerOne in my mind. And when COVID happened, things started to shift both in terms of like spend and priorities and decisions. You know, things had to go virtual. So like live hack events became virtual and, and they were already starting to sort of cut certain costs on things like, Hmm. Man, Angry Blue. Angry Blue posters. Okay. Like behind you.

[00:08:01.00] - Justin Gardner
Okay. Oh.

[00:08:01.33] - Joel Margolis
Like that poster on the left. Angry Blue poster. If anybody's gotten a recent life hacking event poster, that's not an Angry Blue poster. Back in the day when HackerOne first started doing life hacking events, there was this artist named Angry Blue and they would contract him to make one-of-one posters. They were silkscreen printed, they were custom designed. They were made by hand, they were numbered. And like, you could tell that there was like a level of dedication and human effort in being like, okay, we want to make something really special and unique to this. And very, you know, we care about what's going on here. Over time, that shifted. And I think, you know, it's maybe a little cliché, but I do think like an inflection point for HackerOne was when they stopped doing Angry Blue posters and they started doing these like—

[00:08:51.45] - Justin Gardner
Yeah.

[00:08:54.00] - Joel Margolis
laser printed, like very like kind of bland. Like, you know, we hired some Fiverr graphic designer to like, you know, whatever. And like, you know, it was just like the paper quality was cheaper, the design quality was cheaper. Like everything just felt like a little more manufactured. All right.

[00:09:14.30] - Justin Gardner
All right.

[00:09:14.97] - Joel Margolis
Yeah. Yeah. Sorry.

[00:09:15.92] - Justin Gardner
Let me, let me, let me pause you for a second, man, because I've got a lot of stuff to unpack there. So, first of all, one, you know, I just want to also give a little piece of context to the community. Joel and I met through these life hacking events. A lot of the community that you have in Critical Thinking is trying to take what we got at the life hacking events and put it into an every-week podcast, every-week community, right?

[00:09:37.09] - Joel Margolis
I mean, that was one of the reasons we started this was to try and recreate the kind of hacker-to-hacker conversations that happen all the time, like in one-off little like, oh, like, hey, what are you looking at? Oh, do you know anything about this? Like, these things happen all the time at life hacking events. so difficult to have those conversations in, in like spur of the moment, kind of like online communities and stuff like that. They do happen, but, but like the goal was to try and recreate that sort of environment where people could listen to this and they could have the sort of inspiration and the sort of like, you know, mindset and the insight and all that kind of stuff that you get from talking to other hackers at these, at these events.

[00:10:15.13] - Justin Gardner
Yeah. Yeah. And, and it's beautiful and it's magic and it's what has brought The magic to critical thinking that you guys experience is really, to be, to be fair, rooted in those experiences that we had at the Hacker One Life hacking events. I think also, Joel, for you, this is a very, like you said about your post in the HSM chat, it's a, it's an emotional thing, right? Because it was like something that we all hold very dear. And what I, what I want to ask you here is how much of this do you think is the unfortunate lifecycle of companies? And we had the opportunity to be a part of a grassroots, you know, you know, boots on the ground, early stage, right, piece of IT security magic, right?

[00:11:07.83] - Joel Margolis
Yeah.

[00:11:09.21] - Justin Gardner
And versus how much of this do you think is Maybe mismanagement or like neglectful giving up of the soul of a company.

[00:11:24.12] - Joel Margolis
Yeah. And listen, for the people who are like running HackerOne and like the founders and the people who are like driving the business and trying to make this thing successful, like, I really, I sympathize, right? It is not easy to run a business. It is not easy to keep a business going. You have to make it profitable. You have to Mm-hmm. You have to pay the bills, you have to pay the employees, you have to like figure out how to make this machine get filled with gas. The engine has to have gas. You have to figure out how to make that work. And there's, you know, there's a bunch of strategies to do that. I don't think that HackerOne necessarily did all the wrong things or all the right things. I do think you're right that like probably a decent part of this is just like we were there at the right time in the right moment. We got to experience this like really early on when, when there were like less strings attached and they were able to sort of have the freedom to do these things and whatever. But I do also think that there were mistakes in sort of the business structuring and the positions that they, that they decided to take over time and how they decided to move the product and, you know, where they decided to put resources and all that kind of stuff. And, you know, that's not, that's not to say that you know, sales is bad or anything like that. But I think like it's important to remember the core, right?

[00:12:50.64] - Justin Gardner
Yeah.

[00:12:50.75] - Joel Margolis
Like it's important to try and stay true to that core. And, you know, I think part of that is just that a lot of the people who were involved at these stages of early growth and driving these sort of grassroots feet on the ground sort of life hacking events and all this kind of stuff, over time they left and people had to replace them. And those people didn't have the same sort of understanding or experience and stuff like that. And so things just shifted and they moved in a different direction. And the same is true for hackers, right? Like, Yeah. I attended, I don't even know, probably 30 or more live hacking events in the early years. And yeah, I haven't been to one in quite some time, but, you know, I talked to a lot of hackers. I talked to a lot of OG people who were there at the same time I was. And, you know, things are definitely definitely different. I do know that things are different. I can tell both from the outside and from the sort of inside conversations that I've had with people.

[00:13:51.47] - Justin Gardner
And, um, the product still has the spark though, man. I, I'll tell you, at, at recent life hacking events, you know, well, and things— and we'll get into this, but things do shift a bit with AI, right?

[00:14:02.65] - Joel Margolis
Yeah.

[00:14:03.00] - Justin Gardner
You know, one of the situations that we're running into with top hackers, myself included, is you'll get a show and tell, and when you got a show and tell before, you were like, Hell yeah, bro. I'm gonna like show them how amazing this bug is. And when you get to show and tell now, sometimes it's like, oh shit, I gotta go like read up on this bug. Because it's like, you know, the AI did so much of it. Right. And then you're like, oh shit. Like I can't fully explain this. Right. You know?

[00:14:26.89] - Joel Margolis
Yeah.

[00:14:27.27] - Justin Gardner
Yeah. And it's like, and it's like, oh man, this is a very different situation than if you asked me anything, anything about the scope that I had hacked on in the past. I would be able to give you an exact answer off the top of my head, you know, no hesitation, because that's how much like in it, the flow state you were.

[00:14:48.01] - Joel Margolis
Yep.

[00:14:48.57] - Justin Gardner
Right. And nowadays it is a little bit different and results, you know, admittedly results are different as well. We are getting bugs that we never would have gotten before and we are getting a volume that we probably never would have gotten before.

[00:14:59.62] - Joel Margolis
Yeah.

[00:14:59.87] - Justin Gardner
But it hurts at what, you know, at what cost to the soul?

[00:15:03.78] - Joel Margolis
Yeah, for sure. And like, listen, I don't want it to seem like I'm super anti-AI because I think AI is very powerful and I use AI quite a bit. And I think people should be using AI. AI is an accelerator, right? It's a tool that's in your hands and you should be using it. You know, it would be like if, you know, computers were just released and you were like, nah, I'm going to do everything by hand.

[00:15:29.50] - Justin Gardner
Right.

[00:15:30.12] - Joel Margolis
I'm not going to use a calculator. I'm going to, you know, like, You know, the tools are there. You should use them. You should leverage them. They're there to accelerate. They're not there to like necessarily replace you. Um, but, but yeah, it has created a lot of these, these sort of problems, like you said, where, you know, it used to be almost like a, like a craft, right? Where you would like spend this time like doing recon and understanding like very like deeply and like narrowly different parts of the scope. And you would, you would deep dive into these different things and you have all this nuance. The speed at which an AI can do like 90% of that is remarkable. And it's changed things, right? Like you, you don't have to necessarily do that anymore. You can just be like, hey, like go do this stuff for me and it'll take like 10 minutes and it'll be like, here you go. And that would've taken you like 10 hours, right?

[00:16:18.26] - Justin Gardner
Yeah. It's crazy, man. I mean, like literally the other day, I, I, I, I knew that the world has changed when the other day I, Instead of adding a new user, giving it passwordless sudo, and then curling this thing to install it. Right. I literally told Claude, do this, give it passwordless sudo, create this account, you know? And I was like, it's faster for me to just do that, which is nuts to me. Like, soon I'm gonna forget Unix syntax, bro. Like, you know, Bash, it's like, oh my gosh.

[00:16:49.33] - Joel Margolis
Yeah. And like, there's always that joke that like, you know, like, I don't know, like 10 years ago it was always like you'd Google something and then you go to Stack Overflow and just like copy paste from Stack Overflow. Like, uh, these This is sort of the modern equivalent of that. Like you go to AI and you go, hey, like, how do I do this thing? And it'll be like, yeah, just run this command. And like, that's, that's okay. Like, that's fine. Like, um, you know, it's a resource and it's, and it's there to be used.

[00:17:12.19] - Justin Gardner
Um, but yeah, I agree with you though. I mean, with each, with each hop, the live hacking event has changed. The experience for the hacker has changed. And one of the inflection points that you mentioned in your blog post was the, um, uh, you know, instantia— I want to say instantiation. What the heck? I'm such a— well, hey, all of you guys know what I'm saying. The instantiation of the, um, HSM program, right?

[00:17:38.97] - Joel Margolis
Yeah.

[00:17:39.54] - Justin Gardner
Um, and, and I, yeah, I want to, I want to ask about that because, yeah, I hear a lot of people saying, look, it's really hard for new hackers to come up and, and get there. And, and on one hand, I, I hear you. On the other hand, write good reports with good POCs and, and submit them, and you will get bounties and you will get, you know, invites to private programs.

[00:18:06.36] - Joel Margolis
Yeah.

[00:18:07.38] - Justin Gardner
Uh, you know, and it is, it is a learning experience. Nobody's going to write perfect reports every time. But the one counterexample that I always point to is Space Raccoon, dude. Space Raccoon came up when I was already established in, in the community. And I see this guy and he rockets to the top of the leaderboard.

[00:18:22.58] - Joel Margolis
Careful, he's right behind you.

[00:18:24.09] - Justin Gardner
Yeah, yeah, exactly right. And, you know, and he's just nailing it, right? And then he, you know, does his thing, right? So, and there's countless examples of this today— Excess Doctor, Bust, like name all these people that have come on the scene in the past year or two. Um, and they're— it's, it's still able to be done. So I wasn't really sure what your beef was with the HSM program because, yeah, you know, when they— when you are an advanced hacker, you know, when a report is not being handled correctly versus when it, you know, versus when you think every report is being handled incorrectly because you're inexperienced. So then you gave them extra escalation privileges, right?

[00:19:01.36] - Joel Margolis
Yeah. Yeah, for sure. And I think— I don't think the HSM program is like something that needs to go away. I think I really do think the HSM program is extremely valuable. Where I do feel some conflict is in a couple of areas. One is that You know, as I mentioned, I do think it is a little unfair in terms of the advantages that you get by being in HSM, right? I'm, I'm in the HSM program. Uh, I have somebody who works at HackerOne who I can send a DM to any time of the day and they can move mountains.

[00:19:37.25] - Justin Gardner
Mm-hmm. Yeah.

[00:19:38.27] - Joel Margolis
If I'm a new hacker and I, I could be having the same exact experience as a top hacker, I could have a completely valid report. I could be struggling with this company. I don't have that kind of resource to be able to be like, hey, can you help me with this? And have them like go reach out to the program manager. That it's just, you know, and anybody who's worked with HackerOne support knows it is not an amazing experience when you try and do that compared to like the HSM that you have to be like, hey, like, uh, can you help me with this? And they'll be like, yeah, sure. One sec. Let me reach out.

[00:20:06.32] - Justin Gardner
Um, but you can't offer that to everyone. So it's like, you know, right. Yeah. So I just don't want you— this is my beef with the blog post, Joel, my number one beef with the blog post. I agree with most everything that you said in there. But I don't want you to feed the voices of those little whiners that say it's impossible to rise up in bug bounty. It's not. Yeah, for sure.

[00:20:25.77] - Joel Margolis
For sure. And like, yeah, like, I don't want to— it's impossible to scale this, right? You can't just be like, oh, every hacker now has access. Like, it's— there's a reason that it is this way, right? Like, the goal is to make the experience the best for the people who are, you know, doing it outside. Right, right, right. Like, and I don't want to imply, you know, that like, you know, new hackers aren't providing value or anything like that. Like, it's just, there is a sort of like proving ground that is required to get there. Okay. And like, that's fine.

[00:20:58.07] - Justin Gardner
I would even challenge you on that, man. Like, you know, the hackers that are established, that are churning this, that are full-time hunters, we are adding the most value and that's not a problem. You know, the newer hackers add value as well. But when you look at it at scale and you look at it at experience and what, and what wisdom goes into deciding, okay, looking at this app, consuming the vibes that come from the policy and the threat model of the app, and then giving you a, you know, report that aligns with a violation of your threat model. That is an art, bro. You know, that really is an art. Right. And some people are more experienced and thus are going to be able to produce more output.

[00:21:35.23] - Joel Margolis
Right, right. Yeah. So, so, you know, I, you know, I kind of threw that, that piece in there in the blog as sort of just like, You know, HSM's not, the HSM program's not perfect.

[00:21:45.38] - Justin Gardner
Mm-hmm.

[00:21:46.54] - Joel Margolis
What, one of my bigger sort of complaints with the HSM program is that the HSM program should be, if nothing else, one of the most tailored to hacker parts of like the whole business.

[00:21:58.29] - Justin Gardner
Mm-hmm. Yeah.

[00:21:59.15] - Joel Margolis
And when it comes to like feedback and stuff from people in the HSM program, that is stuff that should be taken as seriously as stuff from customers, if not more. Okay. These are people, these are the people who are using the platform the most. They are doing the most legwork. They are interfacing with stuff on the platform the most. And when they have some feedback that we should be doing something with that, get it done. And this plays back into the AI stuff, right? Like, if, I mean, anybody who's done anything with AI, you can add features in the blink of an eye. You can fix stuff. You can change stuff. You can do stuff so much quicker and easier than you could. And It is beyond frustrating to me when I see people be like, hey, uh, you know, I have, could we get this feature added? Or, you know, I'm having this like weird problem with the thing. And like, people will like give it a thumbs up. They'll give it a plus one. And then silence.

[00:22:53.42] - Justin Gardner
Nothing happens.

[00:22:53.79] - Joel Margolis
Like nothing, nothing happens. Right. Like, I mean, when that comes in, you should just be like, hey, Claude, go do this. And then like in an hour you should be like, okay, this is done. Like, I can think back to the early days of of HackerOne live hacking events when the founders would be there and you would have a conversation and be like, hey, like I'm experiencing this weird thing. And Yobert would stay up all night implementing that feature by hand for you. And he'd come back to you the next day and be like, hey, this is fixed now.

[00:23:24.16] - Justin Gardner
Yeah. I freaking love Yobert, man. Oh my God.

[00:23:25.83] - Joel Margolis
Like we should be doing that. Like it's not hard and it's not like an outsized amount of resources. If anything, it's less resources to just do these things. And it would, it would serve the community and it would be really easy win.

[00:23:38.83] - Justin Gardner
Um, couple, couple thoughts on that, man. Like, I totally agree with you. And especially in an environment like this where the power user is actually just a scaled up, uh, normal user, right? Like there's, there's, you know, there are some softwares. Uh, all right. I'll, I'll, won't beat around the bush. I got a, I got an earful from the Kaido team because I go to them and say, hey, I need this little thing fixed. And they're like, Justin, you are a top 0.01% user of Kaido. Okay, we've got this thing that we need to fix for the masses and you're up our ass about this, right? And I'm like, okay guys, okay, okay, okay, you know?

[00:24:15.47] - Joel Margolis
Yes.

[00:24:15.78] - Justin Gardner
And so on one hand, I do understand situations where power users, while they are very important, they are not necessarily the highest ROI things to fix. However, in this situation, HackerOne power users are very, very similar to normal users, except that they do it more. You know, it's not that they're using it completely differently or anything like that, right? They're just doing it more.

[00:24:41.96] - Joel Margolis
Yeah.

[00:24:42.27] - Justin Gardner
Um, and so if you have feedback and a wider variety of experiences that are being, you know, given, uh, feedback by the, the power users, then that should go straight to head of engineering and get shot down. I, I mean, right? Shot down to the team. Yeah. Not shot down, shot down into the code. Yeah.

[00:25:00.94] - Joel Margolis
Noted down. Exactly.

[00:25:02.90] - Justin Gardner
Yes. Yes.

[00:25:05.16] - Joel Margolis
And I feel that. Yeah. I mean, not, not to, not to lean into HackerOne too much here, but, uh, when we're talking about resource allocation, okay, I know HackerOne has engineers. Okay. I met them. I know they exist. I know they're there. I know they go to work 9 to 5, Monday through Friday. I know they're, they are employed. So what the heck are they doing?

[00:25:32.78] - Justin Gardner
Yeah.

[00:25:33.50] - Joel Margolis
I mean, what, what the platform has fundamentally stayed the same for 10 years, right? The new features are in like one bucket, AI, right? We've got, we've got Hi, we've got like report writing, you know, we maybe added like templates.

[00:25:53.76] - Justin Gardner
Okay. I will say, I will give credit where credit is due, the AI-powered, um, data analytics on your own submission statistics. That is nice. I really like that. Shout out to the HackerOne team for that.

[00:26:06.19] - Joel Margolis
When I was on the program side, that was like the most useful thing that, that they had added in years was the ability to just be like, hey, um, uh, like, where are we seeing like the most number of reports? Or like, give me some, some like stats about like, you know, average time to response and stuff. And like the ability for it to then just like look at the data and give me the— awesome. That is a Great example of like how we can actually use AI to like benefit programs and hackers and everybody and just like give them this information without having to try and build some like feature to do that because like graphs are hard and like, you know, just ask the AI and give it the data and whatever. Like, awesome. That's like one thing. Like, what, where's everything else? Right. And I'm biased, you know, I can think of one very specific example where I was like, hey, could we add Uh, can we add like a filter to this specific page? I implemented it myself. This is like GPT-3 days. I implemented it myself through TamperMonkey in like an hour with ChatGPT. It has still not been done. It's been like a year and a half, maybe 2 years.

[00:27:06.05] - Justin Gardner
Dude, is it the submission when you're selecting the CV?

[00:27:07.98] - Joel Margolis
No, no, no, I can't. No, but it's like, you know, it's one of the listing pages, you know, we'll say like, you know, you're looking at a listing of different things like programs and, you know, I asked for a filter to be put into one of these pages. And stuff like that. I mean, Claude could have done this in like actually 10 minutes now, right? Like this is, um, you know, it's not a hard feature. And, um, again, like I think when you're receiving feedback from your power users, like, yes, it's probably a little more niche, but they're also some of the easiest wins that could be done. And when I contrast that with like other stuff on the product, like if the platform was like constantly changing and they were like pushing out new features like every 2 weeks and stuff. Like, I'd be like, okay, like I get it. They have other stuff going on. Like, like the Kaido example I think is perfect. Like Kaido, the way that they manage feature requests is that they put it on GitHub issues and they say, go give it a thumbs up if you want this to be done and we're going to handle it from like highest number of votes to lowest.

[00:28:05.72] - Justin Gardner
Right.

[00:28:05.97] - Joel Margolis
Like there's, I mean, probably there's an internal tracker for that at HackerOne. I don't know. I would hope so. But like, The community tracker is basically like people giving the +1s to like messages and stuff. Like if you see it and like, that seems to basically go nowhere.

[00:28:21.82] - Justin Gardner
Um, and you know, yeah, I think it just also is a function of like, you don't get feedback from the users that don't stay using your product too sometimes. So like what I'm trying to talk to some of the people I advise for, not just Kaido, others as well. But it's, it's a situation where you're not gonna get feedback from a user that's not invested in your product. So let me give you the feedback that might be costing you conversions in the first place. Right. And, uh, and yeah, I don't know.

[00:28:54.48] - Joel Margolis
Yeah.

[00:28:54.57] - Justin Gardner
I feel like a lot of these are small, but also Joel, I mean, look, I, I mean, you know better than I do cuz you were actually an engineer on it, on this stuff. But I, I, I underestimate the intensity of engineering tasks. sometimes, but also sometimes I definitely don't with AI, you know, like, so, I don't know.

[00:29:13.50] - Joel Margolis
Yeah. Yeah. I mean, the, the tool to, like, if we'd been having this conversation 5 or 10 years ago, I get it. Like, you gotta put, you know, you gotta put this in the sprint, you gotta allocate some points to it, you gotta figure out how this is gonna fit into the development timeline and stuff. But again, like, I would be significantly more understanding if the platform is constantly like evolving and changing and doing and adding new features and doing all this stuff. And I'd be like, okay, like, I get it. They're working on other stuff. I mean, correct me if I'm wrong, but like, I, it's, I have seen no noticeable changes anywhere except for in like the, the, the high bucket. And I'm like, okay, like, what are we doing? Like, really, it's like the whole engineering team, like working on like improving high, like every day. Like, is that really like where all the time is being spent? I don't get it.

[00:29:56.71] - Justin Gardner
The mobile interface as well, dude.

[00:29:59.23] - Joel Margolis
Oh my God. Oh my God. Don't get me started on mobile.

[00:30:02.20] - Justin Gardner
Yeah.

[00:30:02.44] - Joel Margolis
Go to just open HackerOne, sign in. If you can, I mean, you know, it's like 3/4 of the screen is taken up by like a sidebar and like you can't, you can't navigate the inbox. It's impossible. Like you got to turn your phone landscape to even do anything.

[00:30:16.88] - Justin Gardner
Or turn on desktop mode and zoom all the way out.

[00:30:19.28] - Joel Margolis
Right.

[00:30:20.15] - Justin Gardner
Um, so we've been saying HackerOne, HackerOne, HackerOne, HackerOne here. Yeah. Like, is this because HackerOne has wounded you deeply in the heart because you were their poster boy on their homepage and now you're You're mad at them because they don't give you nice posters or, or is it, or is this, you know, I mean, surely in part it's a little bit that, right? But I, but I, but I think also, to be fair, a lot of the other platforms are dealing with similar issues.

[00:30:52.30] - Joel Margolis
Yeah. Yeah. Yeah, for sure. And like, I don't want this to seem like it's a HackerOne only problem. I wrote the blog post about HackerOne because HackerOne is who I have the most experience with. Uh, I, I am on the other platforms, but I do almost no hacking there. I probably should, but like, HackerOne has just been sort of where I spend my time and I put, I put my energy and I have the most, um, context and face-to-face experience and real-world experience and all that kind of stuff with HackerOne.

[00:31:20.00] - Justin Gardner
Which shows the value, by the way, of what they've done. Right. You know, like I've also felt this, like I've wanted to hack on Bugcrowd more, or, you know, or, or Yes We Hack and Integriti more. I just, I just, you know, I don't know. They've like, HackerOne is, is they've got their hooks deep. You know what I'm saying?

[00:31:39.91] - Joel Margolis
Yeah.

[00:31:40.05] - Justin Gardner
I don't know why. I don't know why. And I have shifted. I've gone, you know, hacked on Google, done a lot of Bugcrowd this year, you know, but it was like, you know, I was really struggling to move away for no reason. Yes.

[00:31:53.78] - Joel Margolis
And I think the other platforms also have a lot of similar problems. I mean, Bugcrowd's interface has barely changed as well. I mean, what's going on there, right? Like, I mean, every, all the platforms I think are struggling with some of the same things, which is like how to allocate their resources and their time and like where to focus and spend time on things. And like HackerOne is not exclusive here, but I spoke about HackerOne specifically because they're the ones who I can have the most like real-world, tangible, direct experience with. And I could, I originally, I actually was going to talk about the other platforms and I was like, I don't think it's really fair for me to, to make these sort of claims and talk about this stuff for the other platforms, because I just, I can't confidently talk about this stuff because I don't have the same kind of experience with them as I do with HackerOne. Um, so I don't want HackerOne to feel like 100% singled out here, uh, even though they were.

[00:32:50.99] - Justin Gardner
Yeah. Um, but yeah, but, but You know, this is probably true for the others as well, but—

[00:32:56.77] - Joel Margolis
Yeah, right, right. But like HackerOne was also like one of the— I don't know when Bugcrowd was started. Probably should. I probably should. But like it was probably around the same time. But, you know, HackerOne has been in this space for quite a bit of time. And, you know, I do feel like they—

[00:33:14.22] - Justin Gardner
2012, same time.

[00:33:15.20] - Joel Margolis
Yeah. So right. Yeah, same year. Right. So you know, they both had the same amount of time and opportunities and the technology and all that kind of stuff to, to like really shape their, their platforms.

[00:33:28.06] - Justin Gardner
And yeah.

[00:33:29.69] - Joel Margolis
Yeah.

[00:33:30.48] - Justin Gardner
Yeah. It's, it's been, you know, I guess it's a double-edged sword, you could say, with the intensity of the relationship that they've built with the hackers, right? Because I think that when you do build an intense relationship, you get strong advocates and you get strong loyalty. But then you also get strong feelings when things change, you know?

[00:33:49.26] - Joel Margolis
Yeah.

[00:33:49.97] - Justin Gardner
Um, it's an interesting, it's an interesting case for a tech company that requires strong, you know, continued engagement in this, in this, in this way. Right. On, on one, how to do that. And then what, what can happen if that changes? Right.

[00:34:10.01] - Joel Margolis
Right.

[00:34:10.26] - Justin Gardner
But, but I will, I will say, Joel, as well.

[00:34:11.96] - Joel Margolis
Yeah.

[00:34:13.21] - Justin Gardner
I, I'm also not super big into the whole like Ambassador World Cup or like some of the other stuff that they're doing to, to bring new hackers on. But I don't feel like they are not trying to recruit and, and, um, you know, uh, pour into new hackers.

[00:34:30.46] - Joel Margolis
Right.

[00:34:30.84] - Justin Gardner
Um, I think they have a lot of initiatives going for that, uh, and even initiatives that escalate people to the live hacking event-esque level of like, okay, guess what? You won in this like super dupe fest competition. Like, now you get a flight to wherever.

[00:34:47.76] - Joel Margolis
The only reason I ever started bug bounty was because of the HackerOne CTF. Like, I, you know, I don't want to minimize that. Like, community events and stuff are like the way to find talent and bring people into this space and make them feel welcomed and encourage them to like pursue this as like either a hobby or a career or a side gig or like whatever. Like, This is, this is how you, you bring people in. You got to like show them that it exists. You got to like nourish that talent and like keep them interested and give them like the opportunities and all this kind of stuff. Like it's important, right? And I don't think that they've like 100% like done like step back on that or anything like that. I do think like there are definitely pieces of that where it's like, you know, like the regional clubs and stuff. I mean, like I, I don't know what's going on there.

[00:35:42.55] - Justin Gardner
Yeah.

[00:35:42.98] - Joel Margolis
You know, like, but yeah, I mean, you know, I do think that they are, they are trying to do that. I'm not going to speculate as to what the reasons are for that. You know, I'm sure some of it is for the business and, and, but, but like, you know, I'd like to hope that it's, it's genuinely like, okay, we're trying to like grow the community and like, and find new talent and get young hackers into this space and, and give them the resources and stuff that they need to be successful.

[00:36:09.76] - Justin Gardner
So, so let me ask you this then, man. Like, I don't know, I kind of want to save this, this conversation for the end of the episode. But what I plan— I'll give you a little— I want to let something stir with you a little bit here. We're talking about a lot of the problems here. But I don't see a lot of alternatives, you know, a lot of, a lot of different solutions. So I'm going to come back, and you do put a conclusion at the end that addresses some of these things. So I want to come back to that. But one of the other problems that you brought up was their sales, sales incentivization structure, or the way that they've done sales.

[00:36:51.11] - Joel Margolis
Yeah.

[00:36:51.80] - Justin Gardner
And, and why do you think that that is not good? I mean, getting people on the platform, getting them engaged, getting them receiving vulnerabilities, making, makes them more secure and powers our community, right? So, yeah, yeah.

[00:37:06.94] - Joel Margolis
So, so I have a couple disjointed thoughts around this. Okay. One goes back to sort of what I was talking about around like allocation of resources. Okay. Like development of new features and stuff like engineering, all that kind of stuff versus sales. To me, it seems very clear that The emphasis has been put significantly more on sales than it has on engineering resources to like build and develop the platform and create new features and like push like the technology, the core technology forward versus the lines of business in order to keep things going. And again, like, yes, every engine needs gasoline. Okay. You got to have money. There's a reason that Amazon.com is a losing business, but AWS is the core, right? You have to diversify. You have to have ways to like make money and drive other parts of the business and stuff like that. And I get it. I think that the emphasis on sales has created other problems around like quality of programs and the overall sort of like program ecosystem of like the directory of like things that are available. I mean, I get flooded with invites for programs that have like very little or like no scope, like a couple pages. And I know that they're paying HackerOne like a bunch of money in a yearly contract for this. I've been on the contract side, like from the program side. I know like, you know, things have shifted, the like business structure in terms of like how they get money and all this stuff. Like it used to be probably incorrectly, that like you would basically, they would basically make money based on how many bounties you paid. And like, that's not great.

[00:38:54.40] - Justin Gardner
Mm-hmm.

[00:38:54.78] - Joel Margolis
'Cause that implies that like, you kind of need like vulnerable stuff to like keep paying the business or whatever. Like, okay, like whatever. But then like, you know, also, you know, it's hard to scale that. And so I understand like, you know, the annual contracts and all that kind of stuff, but I also don't think that like they need to be like constantly signing new companies and stuff like this and then sending out these, these invites and all this because like, I'm sure as a top hacker, like you also like just get inundated with like invites for a program. You're like, what am I going to do with this? You know, like, I don't— there's like 2 pages. They've got like a directory and like a sitemap. Okay. Like, yeah, they're offering $3K crits, but like, what am I going to— like, there's nothing to be had here. Like, they have no infrastructure. And like, for HackerOne, that's like probably helping to fulfill the baseline income for the company. Like, you know, and I get it. It is a bit tone deaf to me when I see stuff like that LinkedIn post that I included where they're like taking their top salespeople and bringing them to Turks and Caicos and being like, you know, yeah, we, you know, these are our top sales stars. And I'm like, okay, but like, how about the platform? How about the hackers? Like, I get it. Yeah, they brought money into the business and stuff. That's cool. But like, so do the hackers, right? Like the hackers, like hackers create reports, reports create value, value creates necessity for the companies to stay on the platform and continue to see value there and not want to switch or move away or like close their bug bounty program down and put, put resources towards their program and stuff like that. Like, It's, it's not all like one or the other, right? But it does feel outsized to me.

[00:40:40.76] - Justin Gardner
Let me, let me ask you this, Joel. Do you feel like we're suffering as hackers? Like, like, like I, maybe I'm in my, my, you know, H1 golden boy, like happy, happy land, but dude, I wake up every day and I say, holy crap, I, I can go pick any company. And I can hack on them and I don't have to do any marketing. I don't have to do any accounting. I don't have to do anything. I just, I, I just go and I pick up a program and I hack on it and do the thing that I love. I submit a report and money shows up in my bank account. Right?

[00:41:18.55] - Joel Margolis
Yeah.

[00:41:18.94] - Justin Gardner
Like, and sure, there's a freaking, you know, I don't know, am I like, but, but also that is just the kind of person that I am, to be honest. Like, I often go into these situations and I'm like, wow, dude, this is great, you know, and I don't have a lot of like, criticality. I'm not, ironically, I'm not doing a lot of critical thinking about, about this stuff. So I don't know, man, I'm inclined to say sure. And maybe, you know, that, you know, resources are misallocated. But I also feel like I am not over here suffering as a result of that.

[00:41:53.80] - Joel Margolis
Yeah, no, no, I don't think, I don't think hackers are suffering per se, you know, and I think that's a really good point. Like, I don't think you know, again, sales are necessary. You got to, you got to keep things moving. You got to— stuff has to keep going. And honestly, as, as a bug bounty platform, like their goal should be to bring customers on to make them more secure and all this kind of stuff. There's a balance.

[00:42:21.09] - Justin Gardner
Yeah.

[00:42:21.26] - Joel Margolis
You know, and I think my key complaint is really that like, I don't see that same kind of like Is the engineering team getting a trip to Turks and Caicos?

[00:42:29.96] - Justin Gardner
Right, right.

[00:42:31.07] - Joel Margolis
I would love that. Implementing Hai? Like, what's going on? Come on, man. Like, where's, where's their, their trip to the islands?

[00:42:38.94] - Justin Gardner
Like, I also know that you just hate sales in general. Like, like, am I wrong there?

[00:42:44.73] - Joel Margolis
Like, you know, I don't like getting cold called, which HackerOne thankfully has not done to me, but other companies have, and it has definitely pissed me off a little bit sometimes.

[00:42:53.69] - Justin Gardner
Yeah. Um, all right, man. So boiling this down, what it seems like to me is that this, this blog post, it, it has a couple areas where there is, there are authentic and, you know, real critiques of the industry and of HackerOne's decisions.

[00:43:14.48] - Joel Margolis
Yeah.

[00:43:14.88] - Justin Gardner
But I think a lot of this probably is me, you, you know, and, and all of that original squad. Right. That was, you know, I'm not even a part of the originals. You really, I think, are like—

[00:43:25.40] - Joel Margolis
It really is the contrast, right? It's like, listen, like it's a mourning of something we've lost, right? Yes. Like, and change is inevitable. Like, I, I started the blog post out by being like, you probably have no idea what I'm talking about if you started Bugatti in the last like 3 years, because like you, like, if you don't know anything about Acheron and you came in like a couple of years ago, you're like, what the hell is this guy talking about? I don't like, this is. It's just HackerOne. What do you mean? And like, I think for me, a big part of it is that like, it's the contrast between like what used to be and what it is now. And so it's like, that's not to say that like it's completely dysfunctional and it doesn't work and it's horrible.

[00:44:05.01] - Justin Gardner
Right.

[00:44:05.17] - Joel Margolis
Like, clearly it does work and it is working and it's functional and like, that's fine. I want to see HackerOne get some of that OG core stuff back because I think it was such a value to the business. Like, I don't think like, yeah, maybe they lost a little bit of money on it and stuff like, you know, making the custom posters or whatever, or like, you know, creating this sort of environment or whatever. But like, that's okay. That's like, that's, it's about more than the bottom line sometimes. Like it, you know, there is other value that is not direct. There's long-term value in some of that stuff.

[00:44:44.11] - Justin Gardner
Yeah, I think it's so funny, man, because even lately I was working on a program that like they, they want you to hack their devices, right? And, and they'll send you one device and I'm like, guys, I have to break that apart to get it. Like, like, this is such a nominal amount of money to you. Like, why are you not just sending me like as many of these as I want?

[00:45:09.48] - Joel Margolis
Right.

[00:45:09.92] - Justin Gardner
You know, like, like, if I can prove that I need it, like, why are you not just like, this is very little money to you.

[00:45:16.69] - Joel Margolis
Yeah.

[00:45:17.88] - Justin Gardner
So I don't know, man. I agree with you that, that there are, you know, ROI is not necessarily the heart. But I also think that I've been impressed in the past with, with H1's investment into brand. And you're right, maybe this is largely in the past. But, you know, investment into nice swag, you know, and stuff like that. Yeah. And, and I think that that was a good play for them. And I think, you know, because of that, my wardrobe is HackerOne shirts. Yeah. And because of that, let me just tell you this, because of that, my in-laws are, you know, traveling the world wearing a HackerOne thing. And they told me the other day while they were out on an excursion, somebody comes up and is like, oh, HackerOne, like, that's that company. And they got to talk about it. And it comes up, right? Yeah. And that marketing time and time, and they get to tell a story about how their son-in-law loves this shit, you know? And, uh, and that is really, really good marketing for them. Uh, so, uh, one more, one more topic here before we close it out. Uh, unless you have something else, Joel, but what, speaking of marketing, um, there's been a little bit of a difficult back and forth lately with, with HackerOne relating to how they are using AI with researcher reports.

[00:46:30.84] - Joel Margolis
Yeah.

[00:46:31.50] - Justin Gardner
We tried to address the issue with Alex Rice. I feel like that episode went pretty well. We didn't pull punches. He answered it. But then, you know, they keep sticking their foot in their mouth, you know?

[00:46:41.44] - Joel Margolis
Yeah.

[00:46:42.98] - Justin Gardner
Where are you at on that? And yeah, I mean, what— this is actually what I want to ask you, Joel. What can they do, you know, at this point because of so much back and forth and back and forth and back and forth, you know, to give reassurances? And also, if they don't need reassurances, if they're actually doing something that they shouldn't be doing, what do they need to change? Yeah.

[00:47:05.23] - Joel Margolis
Okay. So I'm not going to get into the whole timeline and stuff. You could read the blog. I tried to do a pretty good breakdown of sort of how stuff progressed between February and now. What I will say is that, listen, Okay, if you're— if you work at HackerOne, and you're listening to this, you guys need to align on what the heck is going on. Okay? Because you can't be like, no, we're not using researcher reports. And then like 3 months later, go launch a new product where the second bullet point is like, we're using researcher report data to drive this product. And then when people are like, wait a second, you're using researcher report data, be like, oh, no, no, no, we're not using researcher report data. Figure it out. Yeah.

[00:47:51.78] - Justin Gardner
Yeah.

[00:47:52.00] - Joel Margolis
Which one is it? Okay. Like, I don't even necessarily care if you use the research and report data, give people a way to opt out.

[00:47:59.55] - Justin Gardner
Yeah.

[00:47:59.71] - Joel Margolis
Be transparent about it. Be upfront about it. Notify people, give them a heads up. Be like, hey, we're trying to develop this product. We want to work with you guys. We want to, you know, if you want to opt in for certain reports or certain stuff, or you want to completely opt out, like, here's how you can do that. We're giving you 3 months notice, like whatever, you gotta own whatever's going on because the back and forth and the lack of transparency and the mixed messaging is like, yeah, it's really hard to be like, oh yeah, okay. Like, I believe them. Like, you know, I, after the Alex Rice episode, I was like, okay, like this seems pretty clear. And then like a couple months later, like flip, like people are leaving because there's like a TOS change and it's like, okay, like, I mean, I don't, I don't know what to believe anymore. And like—

[00:48:48.25] - Justin Gardner
Let me just read this line, guys. I think they did end up pulling this back, but it says, like, right after we had this whole conversation, it says, uh, for this agent, testing is sharpened by using context from HackerOne's 12 years of real-world vulnerability data and your prior H1 bounty findings.

[00:49:05.01] - Joel Margolis
Right.

[00:49:05.65] - Justin Gardner
And I was like, you know, it's funny, if they had just had that first part, you know, they'd be like, okay, you know, we like did some data analytics or whatever on the findings and give credit to it. But the, them adding, and your H1 bounty findings is just freaking hilarious to me, man.

[00:49:20.42] - Joel Margolis
Right, right. And like, I've had conversations with, um, PMs and founders and stuff about this. And, um, probably the most frustrating part is that every time I try and have a conversation about this to try and get sort of a clear answer, it devolves into legal speak. And there's a ton of like very specific wording and nuance being used. Oh, we're not training models. Uh, we're drawing conclusions from the, from results on your reports and using that to determine future decisions and shape how they— and it's like, like to me this is like almost the same thing, right? Like, okay, like, yeah, you're not like storing my report and then like feeding that in as a database or whatever. And like you're not creating model weights based on my report data, but like you're taking the essence of my report and whether or not it was valid on that scope, on that program, and then like storing that and then you're using that and like I mean, AIs cheat on benchmarks. I'm not really convinced that it's not going to be able to really infer from that decision, like, okay, here's probably what the original report was.

[00:50:29.03] - Justin Gardner
Right, right.

[00:50:30.63] - Joel Margolis
Again, it's the dissonance between what is said as damage control versus what is then publicized as product features and TOS, written stuff, is Very, very confusing to me on the outside. And you guys just need to align, okay? Like, you need to figure out like what's actually happening. And I don't understand how like a product launch could get all the way to like being public and that wording being put on a page on hackerone.com while it simultaneously not being true. And then like it gets pointed out, it gets reverted, it gets rewritten. And like, Okay, why didn't— who reviewed this? Is it true? Is it not true? Did we just say the quiet part out loud? Or is that like, you know, like what should we have worded it differently? Like, I, you know, and I think that's where like a lot of the conflict comes from.

[00:51:27.51] - Justin Gardner
Or does the left hand not know what the right hand is doing? You know, like—

[00:51:30.78] - Joel Margolis
Right. Is marketing like completely disconnected from— okay. Like maybe you guys should have a meeting about that. I don't know. Like, but, but yeah, I mean, You know, transparently, um, I've, I've had a lot of people who are on the leadership team at HackerOne say like, this isn't happening. And I'm inclined to believe them because they say it. I, I'm not going to call them liars. Uh, what I will say is it's really hard to tell whether what they're saying is nuance or pure fact.

[00:52:03.98] - Justin Gardner
Mm-hmm.

[00:52:04.36] - Joel Margolis
And when time and time again, stuff like this happens, it gets harder and harder every time to really take their word for it and be like, okay, like my data's not being used. And where's the opt-out button, man? Where's the fricking opt-out button? Because I should be able to go into my profile and click on privacy and be like, don't use my report data for anything AI related. Like that should be a thing.

[00:52:27.30] - Justin Gardner
Yeah. And, and, you know, or an opt-in button with some incentivization. You know, that's, that's another thing is like, you know, and Shubs has brought this up before, we would, might consider, you know, like passing along our corpus of reports, you know.

[00:52:43.11] - Joel Margolis
Dude, before AI, we literally had like, you know, these people who had their automation stuff and we'd be like, okay, like I found this bug. Can I, uh, give it to XYZ and you go scan all the programs that you're a part of, submit it to all of them and I get a 10% cut.

[00:52:57.73] - Justin Gardner
Yeah. Yeah, do that. That still happens to today, you know? So yeah, I agree with you, man. I do. I think that that is something that it has been really, really difficult to get a clear answer on. And I hope that, that this helps them align on that.

[00:53:17.61] - Joel Margolis
Yeah. And when I see them do the same damage control strategy, whether it's like, you know, people get pissed and then like instead of a PM coming and saying something, you've got like one of the founders like who's like never sends a message anywhere, like suddenly they come and give like a long paragraph being like, oh, like, sorry, like, this is what's actually ha— it's like, come on guys. Like, uh, I mean, yeah. What, have some internal alignment, right? Like right hand, left hand, talk to each other. Uh, you know, before we're going in and like publishing a product, like let's, uh, review it. I like, yeah. Yeah, I know. It's, um, I hear you. Yeah. Yeah. Frustrating.

[00:53:54.07] - Justin Gardner
It is frustrating. And I think some of the, one of the unfortunate things as well is we haven't seen a lot of deliverance on on some of the takeaways from the episode that we had with H1, with Alex Rice, and some of the other things that we've talked to them about, like creating some communication channel for hackers to express their concern about this and get answers directly. I think that is essential in this day and age to, one, proactively start a conversation. Uh, you know, but 2, there's a lot of fear, there's a lot of disruption. Like, let's try to keep your community calm, collected, and progressing forward through this AI, you know, change that's happening.

[00:54:38.80] - Joel Margolis
Yeah.

[00:54:39.11] - Justin Gardner
Um, and, and in that light, man, I, I did want to swing around to my question earlier, is like, you know, how do you think, how do you think HackerOne can revive their prior identity? Because in one hand, on one hand, You know, hand, we realize that small company vibes are small company vibes, and they feel good, and they're close to our heart, and we feel special to be a part of it. But on the other hand, we've seen, for example, things like Google, one of the biggest companies in the world, you know, get Larry Page and Sergey to come in there and just like go founder mode, and then yoink products out of nothing into you know craziness. Listen to the. What is it, the Acquired podcast, or maybe it was Founders on the Google series. But man, it's, it's amazing. And maybe that can happen again.

[00:55:27.28] - Joel Margolis
Yeah. So I don't want the takeaway from this to be like, okay, we need to fire our whole sales team and like go back to square one. Okay. Like, again, gas, you got to have gas in the engine. Can't run without gas. Okay, I get it. You gotta redo some resource reallocation. Okay. Uh, bring the resources back to the, to the hackers, bring the company back to being centered around the hackers and listen to the hackers, listen to the feedback, action on the feedback. Same for companies too. Okay. By the way, like, uh, when you're a program and you're paying HackerOne tens of thousands of dollars a year so that you can be on the platform and have access to it, And then you're like, hey, uh, you know, can we, uh, have this feature implemented? And nothing happens. It feels the same, if not worse, as when you're a top hacker and you do that and nothing happens. Because not only am, uh, am I like a customer, but I'm paying you money.

[00:56:29.90] - Justin Gardner
Mm-hmm.

[00:56:30.36] - Joel Margolis
As a hacker, I'm not paying you money. I get it if you want to ignore me, but like, as a customer, I'm paying you money and I'm giving you a feature request and nothing happens. That's a really good way to lose customers. Okay. And I get like vendor lock-in and all that kind of stuff. Like that's probably the only thing that's saving them with a lot of these cases where it's like, it's too hard to switch away. And so they just kind of stick with it and they're like, all right, this is what it is. We'll build something else. I cannot even tell you how many programs I know have like basically built their own internal tools around the HackerOne inbox to make this stuff happen for themselves because they just are tired of waiting.

[00:57:03.61] - Justin Gardner
Okay.

[00:57:03.98] - Joel Margolis
Yeah.

[00:57:05.00] - Justin Gardner
Yeah.

[00:57:05.38] - Joel Margolis
So like, there needs to be a focus on features. Okay. Like when, when there's feature requests, there needs to be a timeline. There needs to be like, um, you know, some sort of guarantees. There needs to be some sort of tracking and like, you know, follow through on this kind of stuff. Um, and then like recenter on the hacker. Okay. Make it, uh, like, uh, you, it's called HackerOne. Okay. It's in the name.

[00:57:26.05] - Justin Gardner
The hacker should be number one.

[00:57:28.69] - Joel Margolis
I mean, mic drop.

[00:57:30.21] - Justin Gardner
Look, where's the mic? I don't have a mic.

[00:57:32.07] - Joel Margolis
There you go. There's your, your new company motto. Like, I know that, like, come on, guys. Like, think about the whole impetus of the company. Okay? This was created to create a safe space for hackers to be able to submit and do research and work with companies and create a secure internet. Like, that is the whole point. I get you're a business. I get you got to make money. Do it. That's fine. But don't forget what the whole origin is, right? Like you've got your, you're here to, to create a space for hackers. Okay. It's not just about companies. It's not just about sales. It's not just about hackers. You got to have a balance of all this kind of stuff.

[00:58:12.03] - Justin Gardner
And any ideas on how, I mean, feature requests being heard and that sort of thing, but any ideas on how they can make it feel a little bit more like those magic golden era times that we had in the past? Um, I mean, and I don't know is a perfectly fine answer, Joel. I just— yeah, I don't really know.

[00:58:33.30] - Joel Margolis
I don't know. Um, you know, I don't know. I don't know. And I don't want to just like be like, oh, you got to make the custom posters again and stuff. Like, yeah, I think that would help. I'm not going to lie. I don't think like the amount of money that you saved from not using like Angry Blue or whatever is like really, I mean, you know, it doesn't impact new hackers, but like the guy, the hackers who saw it before, like, this feels kind of cheap. You know, it's noticed. I know it's noticed. I've talked to people. We notice it.

[00:59:04.23] - Justin Gardner
They do. They still do give out high quality swag though. I've got a Stanley mug. I've got a North Face, this, that, you know? Yeah.

[00:59:10.80] - Joel Margolis
So yeah, I've got plenty of really nice H1 swag, but you got, you got to make it feel special and you got to make the hackers feel like, Um, like, like they're, they're, they're the focus. They're, you know, they're the reason why you're here. You're here to serve them and, and, and all that kind of stuff. You got to figure out how to balance that with the rest of the business, you know, keep customers coming in, keep high quality programs coming in, work with the customers, work with the hackers, but you know, it, um, uh, maybe the word disposable is a little too harsh, but it does feel like that sometimes as, as a hacker where it's like, if you don't want to be here and you don't feel valued, somebody's standing right behind you waiting to take your place. Um, which is good for them, but also not great.

[01:00:03.40] - Justin Gardner
Yeah, I feel you. One of the things you said at the end was sort of a call to action for market disruption, um, in the bug bounty world. And there's been some talk of that already with AI-related stuff, um, you know, throughout the community. Any, any thoughts, any words of wisdom, any, um, besides your, you know, the tools are in your hands, build what HackerOne could have been at the end of your—

[01:00:30.13] - Joel Margolis
Yeah.

[01:00:30.76] - Justin Gardner
Um, yeah. Researcher centricity.

[01:00:39.19] - Joel Margolis
Yeah. Yeah. I mean, like, you got to be researcher first. You know, everything else, again, like, if you have a, if you're HackerOne and you imagine like tomorrow, every hacker on the planet vanishes, now you've got a bunch of companies and you provide no value to them.

[01:00:53.94] - Justin Gardner
Mm-hmm.

[01:00:55.69] - Joel Margolis
So it's a necessary part of the equation. Okay. You have to think about the researchers and in the same way that you want to incentivize customers to stay, one of the biggest incentives is to make it so that they get value from the researchers on the platform. That's why they come to HackerOne and not one of the other platforms, right? Because for years it would be like, I'm going to HackerOne because of the researcher base there. I'm not going to the other company because they don't have the same researchers. They don't, they don't have the same quality of reports. That, that's real. I mean, and that's shifted over the years.

[01:01:31.44] - Justin Gardner
Yeah.

[01:01:32.96] - Joel Margolis
So you need to figure out how to get that back and you cannot just like sit back on your laurels and just be like, yeah, we're a big player in this space and just like do nothing. Like that only lasts for so long. Market, the dynamics of market economics will shift. They, you have to stay competitive. You have to develop, somebody will come and eat your lunch.

[01:01:55.73] - Justin Gardner
Yeah.

[01:01:56.28] - Joel Margolis
And so you have to, you know, you have to stay on your toes and you have to like continue to push and provide value and change and develop and Mm-hmm. So yeah, I mean, I don't think that there's a one-stop solution, but I think like HackerOne is in the position to do this. It's not— it's— you could do it in a couple months. It's not like an impossible feat. It's doable. You just have to sort of take a step back and think about like, okay, where are the priorities and where are the big asks and where can we provide really easy value and wins and grow and listen to the community and listen to our customers and all this kind of stuff. It's, um, it's, it's not easy, but it's also not impossible. Um, and one other thing that we kind of skipped over, uh, is triage.

[01:02:45.46] - Justin Gardner
Yeah. Yeah. Let's go back to it. Yes.

[01:02:48.17] - Joel Margolis
And, uh, I don't know if you need to pay them more or like—

[01:02:55.15] - Justin Gardner
Okay, dude, hold on. Before we go into this though, That is an impossible problem.

[01:02:59.21] - Joel Margolis
Yes.

[01:02:59.71] - Justin Gardner
I've, I've talked to a couple of people on this and it's like, okay, if there's one problem to solve in bug bounty, it is how do you do triage? Because with a pro— it's a, it's a chicken and egg problem. It's like, okay, by the time the triagers are skillful enough to produce, to consistently reproduce, you know, reports, they should just go do bug bounty themselves and make more money.

[01:03:22.59] - Joel Margolis
Yep. Yeah.

[01:03:23.44] - Justin Gardner
So, so what are they doing? And so as a result, what you're, what you're left with is, you know, it's already a very narrow set of people that can reproduce these reports. It's an even narrower set of people that value security, like job stability and security, so much to take a triage job instead of hacking in order to, you know, keep their paycheck coming in. rather than risking it and going for higher income, more flexibility, and not having the trade-off. So I think it's almost an impossible problem.

[01:03:57.88] - Joel Margolis
Yes, it is really a difficult problem. I mean, I would encourage anybody if they have a solution to it, go ahead and say what it is because I don't know what it is. And I'm glad you didn't ask me for the solution because I don't have one. I mean, it is such a hard scaling problem. I mean, and again, like you're bringing more programs on, you're bringing more researchers on, you're bringing more reports in, you got AI, you got the volume of reports and stuff that needs to be validated and like growing exponentially. It is such a hard scaling problem, but you're also providing this service and there's an expectation as to what it's supposed to be. And most of what I've heard is that the overall experience between researchers and triage has gone down.

[01:04:42.01] - Justin Gardner
Yeah.

[01:04:42.25] - Joel Margolis
And it already was a challenge years ago before AI and before all this kind of stuff. And it's continued to decrease. And so maybe there needs to be like a complete rethink of triage as a whole and like how this should work. Maybe like, you know, like here's a really good example, Facebook or Meta. They have this whole thing called FBDL, right? They're like—

[01:05:07.51] - Justin Gardner
Yeah, we covered it on the pod a couple weeks ago.

[01:05:09.44] - Joel Margolis
And like, I think that's a really interesting and unique sort of approach to like how to make triage more effective, right? Like they basically have built like a structured language where you can, you know, like quantify like, okay, like I have account A with these certain parameters and account B with these certain parameters, and it goes through these specific scenarios and like, bam, bug reproduced, right?

[01:05:30.55] - Justin Gardner
Mm-hmm.

[01:05:30.84] - Joel Margolis
Like, I don't think that's, you know, there's a lot of nuance to that, but like, maybe that's something that needs to be thought about as a way to like reduce. Because one of the biggest complaints that I hear is like, you know, I wrote this bug, it's really clear, it should be super obvious. Triage comes back, oh, I can't reproduce it. Oh, I don't understand. You know, like, I don't see the value and, you know, the impact or like, you know, I'm not seeing this. Can you explain? And it just becomes this like back and forth, like tedium. And like, you know, we're— we got to figure out some better way to help like mitigate that and reduce that because that's going to make the triage job easier. It's going to reduce the amount of like revisits to reports and like the back and forth conversation with researchers. It's going to like increase the fidelity of like triage outputs to customers and stuff. But like, I don't know. Triage is a really hard problem. It's really hard to like have a workforce that like can stay dedicated, be trained, have the like the knowledge and understanding, be incentivized to like continue to do it and stick with it for like long periods of time. One of the things I said in my blog post, the top triagers almost always just go and do it themselves. They're just like, I have all this knowledge. I've seen the reports. I see how this works. I get it. It clicks. What am I doing here? I'm just going to go make all this money that I'm seeing these other researchers do because I'm, I'm like tied down and locked into not being able to do these things and I can't get the same benefits. I could just go do this myself and they do and I don't blame them, you know, wouldn't you? Right. So it's a really hard problem. You get creative, you got to have the right incentives, you got to have the right mechanisms. And this is a great place to innovate.

[01:07:13.55] - Justin Gardner
Yeah.

[01:07:14.28] - Joel Margolis
Differentiate yourself from the other platforms and the other things out there by like creating a better experience and like a unique way to like reduce the triage friction. Like that's, that's such, you know, an innovation point that I think like whoever manages to crack that egg is going to have a leg up. Oh yeah, for sure. But triage, I mean, triage, triage needs attention. It for sure needs attention. And I don't think just like getting more triagers is the solution.

[01:07:44.46] - Justin Gardner
And they know that and they're, they're trying with AI, right? The HackerOne agent or whatever that we blew up on them for, you know, cuz it's, it's an agent and it's got our data, blah, blah, blah. That's another thing on the AI front. I really don't get like, and I'll, I've said this time and time again to the audience base, but I'll say it again. You know, you're worried about your, your report ending up inside of AI. Probably you already fed it to AI.

[01:08:07.36] - Joel Margolis
Let's just be clear.

[01:08:08.32] - Justin Gardner
For one.

[01:08:09.03] - Joel Margolis
Yeah. For For

[01:08:09.42] - Justin Gardner
For For two, no one gives a shit about your bug bounty report. You know, like, like if they've got access at OpenAI or at, at Claude, you know, Anthropic or whatever, to all the data that they put in there, they're not going in there like, oh, let me find Retorators like I do, or, you know?

[01:08:25.63] - Joel Margolis
Yeah.

[01:08:25.82] - Justin Gardner
You know? Yeah. So really don't worry about that. I mean, sure, you could say the training, but also we're all standing on the shoulders of giants. How did we learn to hack like this? It's because, you know, and if you've used any of the models at all, they're already better than you, you know, like it's crazy. So yeah, um, I, I don't give anybody flack for parsing reports with AI, um, in my opinion.

[01:08:45.89] - Joel Margolis
Yeah, I think the, the only concern is really just like the, the, the usage of the data, right? Like, I, I, and, and I agree, like, I don't even necessarily think it's like, oh, an AI is reading my report and that is the core problem. I think the core problem is like Reusing that data or like taking my novel techniques that I might've found without AI and then like using that to provide value without giving me any cutback. And so like, it's, it's a more nuanced problem than just like report, AI reading report equals bad. Like, I don't, I don't think that is necessarily really the problem. And if you were to like talk to researchers and poll them and stuff, like, I don't think they would all just be like, yeah, I don't want an AI, like even like knowing my report exists, because I think, again, like the number of hackers who are like not using AI is probably significantly less than the number of researchers who are using AI already. So like you said, like they probably have already seen your report or maybe they wrote it for you and like, like, you know, it's, it's there, like, but, but I think it's just like the privacy aspects and stuff that, that are probably the bigger concern.

[01:09:50.34] - Justin Gardner
Yeah. So, so closing all that out, you know, We are putting a target on the back of triage for anybody who wants to try to disrupt in this space. We've seen attempts at it with agents. We've seen with advanced tooling, even a flow that can get humans up to speed on how to triage these bugs quickly and effectively.

[01:10:10.56] - Joel Margolis
Yep.

[01:10:11.19] - Justin Gardner
Or maybe FBDL, shout out to Ads Dawson for making that. And yeah, There does need to be some innovation here. And if I could give my take a little bit on the founder thing, on the, you know, getting the identity back, I personally would love to see, you know, the OG founders of H1, of Bugcrowd or whatever, step up a little bit here publicly in these situations and say like, look, we are, you know, kind of like what they're doing at Google, right? With the plays with Gemini, you know, Bryn is taking over Gemini, you know, and, and we've seen massive changes when that happens. And you guys understand the whole— the, the whole— the heart and soul as a combination word. The whole— no, the— hey, maybe that's, uh, maybe that's saying something. You've seen the whole—

[01:11:02.32] - Joel Margolis
You've seen the whole—

[01:11:04.09] - Justin Gardner
you've seen the heart and soul of the company. And perhaps it needs leadership through this that kind of leadership, not, not, you know, saying that everybody isn't out there doing the best they can do, but it's different when you've lived it and you've been there and you've, you've seen what it was in the past. But also as a founder, I also understand, you know, founder fatigue and you guys have done a lot for us.

[01:11:28.35] - Joel Margolis
Yeah.

[01:11:28.44] - Justin Gardner
And I want to close this out by saying as well, like, holy moly. Like I said earlier, man, like what, what a beautiful thing Bug Bounty is and how grateful I am I am for bug bounty in general, to HackerOne, to Bugcrowd, to Intigriti, to Yes We Hack, to the whole community. Um, because wow, we get to have complete flexibility. We get to make great money. We get to do exactly what we, we love, hacking stuff. We get to do no marketing, no accounting, nothing. Submit the bugs, we get money in our pocket. Yep.

[01:11:59.68] - Joel Margolis
And the mission has been, I mean, the mission has been accomplished, right? The mission was like, make a safe space, make it, make it possible for hackers to like legally, like get paid to, to find vulnerabilities in companies and like build these relationships with companies. Dude, so many researchers like go on to like work at these companies because of how much value they provided. And like, like you've done it. Okay.

[01:12:19.35] - Justin Gardner
Yeah. Yeah. And we, and we are grateful and we are happy. And as much as these things come out, you know, as, as, as critique, I just, I feel an overwhelming sense of gratitude for this, this gift that we've been given and the journey that you and I got to go on together, Joel, of like the beautiful era of, of bug bounty, the golden era, like you said, truly. And the experiences that we got to have, man, if I were, if I were there, I'd give you a hug right now because we got to experience that together and we got to build the friendships that we have and the community that is around us today. Right. Critical thinking, you know, for sure. The homies.

[01:12:53.56] - Joel Margolis
Yeah.

[01:12:54.06] - Justin Gardner
You know, Jason Haddock squad.

[01:12:55.85] - Joel Margolis
Yeah. I like, I can't thank everyone. And I like, seriously, like most of my close friends now are people who I met through the hacking scene, through, through live hacking events. through these community sort of things through HackerOne. And I'm extremely appreciative for that. And, you know, I don't want to, I don't want it to sound like I'm like shitting all over HackerOne. Yeah, I, but I want, I want to see them succeed and I want to see them be appreciated by the community. Yeah. In the same way that we did back in 2017.

[01:13:27.85] - Justin Gardner
And the community get to experience Yeah, 100%. I'm with you, man. All right, dude, thanks so much for coming on, man. It's great. Thanks for having me. Great, great to having you back.

[01:13:36.59] - Joel Margolis
This was such a nice little reminiscing. It feels— it was so natural. Just like how I couldn't believe when you said it'd been almost 5 years. It was crazy.

[01:13:46.47] - Justin Gardner
Yeah. All right, man. You wanna hit 'em with a, that's the pod? Do you wanna hit 'em with that?

[01:13:49.47] - Joel Margolis
Sure. And that's the pod.

[01:13:51.60] - Justin Gardner
Peace.

[01:13:52.31] - Joel Margolis
Peace.

[01:13:53.44] - Justin Gardner
And that's a wrap on this episode of Critical Thinking. Thanks so much for watching. Thank you so much for watching to the end, y'all. If you want more critical thinking content, uh, or if you wanna support the show, head over to ctbv.show/discord. You can hop in the community. There's lots of great high-level hacking discussion happening there on top of the masterclasses, hackalongs, exclusive content, and a full-time hunters guild if you're a full-time hunter. It's a great time, trust me. All right, I'll see you there.