Episode 182: Partial Auth, Hackbot GraphQL, and AI's #1 Mission

[00:00:01.02] - Justin Gardner
Your number one job is to get the JS files. You know, get the JS files, get the JS maps, get the lazy loaded stuff. That is your number one mission.

[00:00:24.89] - Justin Gardner
Alright, sup hackers, before we jump into the episode this week, we've got the This Week in Bug Bounty segment. 3 quick rapid-fire news articles this time. First one actually is just shouting out Yes We Hack and their awesome live hacking event they just did. I love it when they do live hacking events on fashion brands, like the LVMH one that they did a while back, super legendary. Wish I had been able to go to that one. And this one right here at LeHack 2026, was on Puma and they had all their e-commerce scope, their backend also in scope and 214 reports in 32 hours. So very successful live hacking event targeting Puma. So this one looked like a great one and probably the most reports I've seen in 32 hours, you know, in a short window live hacking event like that. So congrats to them on that. Next was YesWeHack saving our butts. I don't know if you guys saw this article, but there was a pretty serious attack towards security researchers that occurred that essentially was trying to lure people into using a POC in a CVE repo. And YesWeHack found it and made it public and disclosed everything. It was a pretty sophisticated attack too. There was a RAT that was hidden inside of a compiled library which shadowed a Python source file inside of a loaded module that was required for one of these POCs. So YesWeHack went in really deep. This is a great read. It talks about how shadowing— they used some of the shadowing functionality in Python to import a .so file instead of a Python file.. And then also they used Mapbox as a, as a domain front for this, right? So they're like pulling data out of this like map file, Base64 decoding it and then execing it, right? So it's pretty nice C2 interface here. So if you want to read up on that, link will be in the description. Thank you, YesWeHack, for saving our butts here. Hopefully not, not a lot of researchers fell for this Trojan that was embedded in this POC. So always good to check before you run stuff. All right. Last is an article from HackerOne, Navigating the AI Wave. This is Tony Lee, VP of Ops at HackerOne, kind of talking about some of the difficulties they've seen with 100% plus increase in submissions, how they're dealing with the volume. They are once again updating their code of conduct saying, please make sure that you are putting clear steps to reproduce and demonstrating impact. And guys, there's no excuse not to be building beautiful POCs with, with AI, really, you should be building amazing POCs. Everything I do nowadays is essentially curl to bash, right? And then it just reproduces everything right in front of them. Because it's just so easy to do with, with, with AI. So definitely double down into that. They're trying to increase their efficiency with triage and they're fighting the battle. So hopefully that we will see some increases in triage time as they're figuring out this whole AI wave. If you want to read more, link will be in the description. All right, let's jump to the show. So, you know, I have to say it is a masterpiece what you've done with the Kaido skill, but I have improvements. I have improvements that I've made. Okay, so check this out. Let me, let me, let me drop this to you in Discord really quick.

[00:04:11.91] - Joseph Thacker
Uh, wait, how long has this been out there? Have I, have I been missing out on this?

[00:04:15.44] - Justin Gardner
Yeah. Well, I put it in a, in a PR a couple of weeks ago. And so here's what, here's what I did, dude, is I, I've been using the Kaido skill for a long time and there's still some stuff with it I didn't love. And so I decided, I locked in and I decided, okay, I'm going to, I'm going to fix all this up. So I did a bunch of A/B testing with it, you know, got it running on a bunch of different cloud. Instances. And I figured out, I think the best way to do this is actually to have it use curl most of the time and then have it pipe, you know, have it just, you know, proxy through Kaido. Right. And then when it's using, uh, when it really, really needs to hand something off to the user, then use Replay. And when the user is like really needing it to do something where it can see and it can play around with it where the user can, then, um, then you do that inside of Replay. But I think the primary, you know, the primary thing this PR does is it just gets it to use curl, gets it to use a curl config file and cookie jars and stuff like that. And then only use a replay and collections when you need to do very granular changes or when you need to hand off to the user. Right.

[00:05:23.13] - Joseph Thacker
Curl can use a cookie jar. Yeah.

[00:05:25.86] - Justin Gardner
Yeah. So it can use a cookie jar and it can use a config file as well that like, so you can just, I built into the functionality where it will just export a request from Kaido into a curl config. And then just reuse that crawl config. So you're not constantly writing out full cookies and stuff like that.

[00:05:41.16] - Joseph Thacker
Right. Dude.

[00:05:41.77] - Justin Gardner
Yeah.

[00:05:41.89] - Joseph Thacker
That's legit.

[00:05:42.98] - Justin Gardner
Yeah. Yeah. So I think that's the best way to go. I've got it sitting in PR probably by the time this episode shifts, you know, ships, shifts, ships, then, uh, it'll probably be, um, it'll probably be in prod, but, uh, yeah, I think, I think it's a step up and I also added some like match and replace rule stuff as well. So it can like automatically generate match and replace rules. To turn on feature flags and stuff like that. It's pretty sick.

[00:06:05.50] - Joseph Thacker
Nice. Yeah, I'll go update, that's for sure. And I'm sure with Fable now, uh, if people cannot, uh, get rejected, uh, should use it even better.

[00:06:13.20] - Justin Gardner
Oh my gosh, dude, I don't know. I like— I— as soon as they shipped Fable the other day, when they got it back online, yeah, yes, I just started getting a bunch of denials from Opus.

[00:06:24.37] - Joseph Thacker
Okay, so that exact same thing happened to JD. I think what they do sometimes is, in prep for, and the fear of, you know, people using Fable for, you know, cyber stuff, they basically upped or re-enabled a lot of their, a lot of their protections, like their, their blocks or mitigations.

[00:06:42.95] - Justin Gardner
Yeah, yeah, I think so. They reset something, I don't know, but I kind of got it to work after a little while. Um, but it, it, there was just something that I had in my notes for from before that it was loading in. You know, that was loading in fine before. And then when they shipped Fable, I would load those same notes in and it would be like denied for sure, you know? I don't know.

[00:07:06.07] - Joseph Thacker
Yeah, it's frustrating. I have been trying to keep it in context, but it keeps switching to Opus 4.8. So we'll see what happens.

[00:07:13.87] - Justin Gardner
Yeah, so we'll see. I don't know, man. I think it might be time for me to switch over to Codex. Like, I've been big Claude fanboy, but I've been hearing amazing things about Codex.

[00:07:23.66] - Joseph Thacker
It's really good.

[00:07:24.30] - Justin Gardner
You know, 5.5 I think is quite good.

[00:07:27.37] - Joseph Thacker
So yeah, and they just, they have like a Cyber 5.5 that I haven't tested yet, but I assume is also quite good. The, I will say their desktop app is like mind-blowingly good. I know that, you know, you, you like remote control and stuff and remote control is like decent, but you have to like remember to use it and stuff like that. Codex by default has every session from every login in the Codex app, also on mobile and on the desktop.

[00:07:50.91] - Justin Gardner
Really?

[00:07:51.50] - Joseph Thacker
Yeah, just by default, you're just like, it's always there.

[00:07:53.42] - Justin Gardner
Yeah. Wow. That's really nice. I do, I really like Remote Control, but like you said, I had to shift away from it because, uh, you can't use multiple subscriptions if you have, uh, RC, right? So I don't know. It's, I miss it. My interface for now is not as good, even though it spans multiple accounts, um, things, but yeah, it's just, there's something really nice about being able to pop open the app. And just see what's going on, you know?

[00:08:20.75] - Joseph Thacker
Yeah, I mean, you could just use it for one or two, like, just like your, your primary one.

[00:08:25.38] - Justin Gardner
Yeah, I could. Yeah, and maybe I'll do that as well. It doesn't hurt to put RC in there, you know, maybe it adds a little latency or whatever, but not too bad.

[00:08:32.76] - Joseph Thacker
Yeah, you want to jump straight to a bug or two?

[00:08:35.66] - Justin Gardner
Yeah, um, I've got an— I've got an interesting one, um, that I can share. Uh, I think I might have talked about this one on— I was kind of going back— I might have talked about this one on the pod before. But this was just such a crazy bug that this exists that I wanted to kind of bring it up, um, for the people again. Uh, and let me, let me go ahead and grab the full report, but, uh, this is the weirdest XSS I've ever found in my life. Okay. So, so get this. It is on, I guess I can't say exactly what target it was on, but it was on this provider that a lot of people use for e-commerce. And, uh, the crazy thing about this was it was a reflected XSS, a standard reflected XSS. Okay. But there was some sort of checksum going on. Checksum. Checksum. Okay. It's checksum. Yeah. I guess it's sum. Yeah. I always say checksum because it's one of those things you read and you don't really say checksum going on with this where, um, the, the name of the product Uh, I don't want to disclose too much about it.

[00:09:44.46] - Joseph Thacker
Okay. All right.

[00:09:46.78] - Justin Gardner
Well, the name of the product, uh, you know, that they're working on is a subcomponent is WPM. And so the string that was reflected had to have the characters W, P, and M at specific offsets in that string. So at, at that, how did you realize that? Well, you know, I grabbed a valid one, right? And then I started tweaking it one by one. Right. And then, so it would result in an error if W was not at index 8, P was not at, you know, index 16, you know, and M wasn't at index, you know, 27 or whatever. I don't remember the exact offsets, but I was like, what is going on here? Right. This is crazy. Um, and then. And then here's the other piece of this. The only way you could get it to do escape the double quote that it was in was putting percent 22 in there, right? But percent 22 doesn't count as one character. It counts as three characters somehow, right? So it's like, okay, that's—

[00:10:49.72] - Joseph Thacker
Well, whatever's doing the counting has just occurred before the rendering.

[00:10:52.85] - Justin Gardner
Yeah, exactly. And so you had to like, you had to keep the string the same length. You had to have WPM at the specific offsets. Your URL encoded characters would count as double.

[00:11:05.23] - Joseph Thacker
Right.

[00:11:05.47] - Justin Gardner
You know, or as like triple the actual value, right? Because it would be percent and then the code. And then you had to figure out how to code golf in like whatever you wanted to execute.

[00:11:19.62] - Joseph Thacker
Right.

[00:11:20.05] - Justin Gardner
With keeping those characters at the specific offsets, right? So eventually, eventually I figured— so I got an alert pretty easy because it doesn't take that much effort to get an alert, right?. But getting arbitrary code execution on this was real rough. And so they asked for it too. They were like, I don't know that you can do this. And so I was like, oh, I can do it. So it took me forever, but I did figure out a way to get full arbitrary, you know, XSS execution in there with some crazy code golf stuff.

[00:11:48.17] - Joseph Thacker
That's insane.

[00:11:49.67] - Justin Gardner
But, and the reason that that ended up working, dude, is because the text was WPM. Right? And the offset, I used import so that the M would line up with the WPM in import. Right? So it was like, oh, it was perfect. If this was named anything else, then I would have been screwed. But it fit. It fit. You know, I was able to escape, assign some super short domain to a variable, and then align the M. So, it hits an import and then trigger the import.

[00:12:29.39] - Joseph Thacker
Do you have any understanding for why in the world those offsets mattered?

[00:12:33.70] - Justin Gardner
I have no idea. I think it had to be some sort of validation check. I think they were hashing something and then replacing characters at a specific offset. So, I don't know.

[00:12:48.60] - Joseph Thacker
Yeah, I can't imagine any reason why that would exist.

[00:12:51.11] - Justin Gardner
But it allowed for XSS on any instance. Of this, any customer instance. That's cool. So it was like massive, massive impact.

[00:13:00.25] - Joseph Thacker
Yeah, dude, they pay well.

[00:13:02.40] - Justin Gardner
Oh yeah. Yeah. I love to reflect back on that one because it was like, this is just such a mind-boggling vulnerability, you know? Like, how does this exist? You know?

[00:13:10.86] - Joseph Thacker
Like, yeah, that's legit. The thing I wanted to mention is not a specific bug, but it's like a class of bugs that I feel like, um, especially using coding agents today, we'll often find. Um, because, and there's been like maybe 3 or 4 in the last couple, like month or 2 that are just really neat. And you might have seen some before, but I've noticed that basically through using Wayback URLs, they'll find like, I don't know, API routes that return some like object, like uploaded object, like a document or a file or a profile that require almost like a guest user or something. And so like when you load the page, it will like generate like some sort of like guest token or guest user token. And then that is something. Yeah, it's, it's some sort of like partial auth basically. I know we've talked a lot about partial auth, but then that partial auth will then still be usable to like either update things on that object or that document or that profile, or access other data through the API that it shouldn't be able to. And just, it felt really weird to me whenever like the third or fourth one came through, and I'm like, okay, this is like a bug class that has almost been like untapped. Because yeah, um, yeah, a lot of times it's like legacy APIs or like legacy, you know, URLs where I don't know, something's hard-coded to like give access when it shouldn't.

[00:14:23.26] - Justin Gardner
Yeah, dude, it's funny you mentioned that cuz do you see that under my AI Hackbot tips section here? Like this, I'm gonna highlight it in the doc. So what I was gonna say as well, uh, was very similar to this, which is like specifically tell your Hackbot to focus on getting some form of auth.

[00:14:42.12] - Joseph Thacker
Mm-hmm.

[00:14:42.60] - Justin Gardner
Right? Like, and, and actually, uh, there's this, what is it called? Uh, I think it's called agentmail, agentmail.to. Have you seen that?

[00:14:50.44] - Joseph Thacker
Yeah, we talked about it.

[00:14:51.09] - Justin Gardner
Yeah. Yeah, yeah, yeah. So did we talk about it on the pod? Yeah.

[00:14:55.11] - Joseph Thacker
So I talked to Sam Curry about using this, like in the Discord that we're both in. And there, and he very quickly, and some other people were like, just set up your own. Like it takes like one, like you can one-shot it with like a Cloudflare God token. And then I did after this, because I use, I use these HTML all the time.

[00:15:12.14] - Justin Gardner
Yeah.

[00:15:12.16] - Joseph Thacker
But if you just have a Cloudflare account and you just tell it to set up your own, like agent email inbox, you can just set up your own on your own custom domain in like 5 minutes. It's kind of crazy.

[00:15:21.77] - Justin Gardner
But anyways, that, that is interesting. Maybe, maybe that's better, but like literally agentmail.to, it's free, right?

[00:15:28.45] - Joseph Thacker
For 3 inbox or something. Yeah. Yeah.

[00:15:30.62] - Justin Gardner
So I literally just said, use this. And then it did, you know, it was like, it was like, okay, great. Um, so anyway, I gave it that, but what I was going to say was very similar to what you were saying, which was like, one, I have noticed the exact same thing, um, telling the agent to go after getting some form of partial auth, like either a guest account or something, opens up a whole new world of scope typically. Um, and I think the agent's pretty good at spotting when a weird cookie gets set on the response of something, right? So I, I think like, like you said, I've also seen in my, in my, um, tests that occurring and then being able to use that same cookie. To like access other stuff. So that is definitely a systemic issue. I think it's funny that both of us wanted to bring that up today and we didn't even compare notes on that.

[00:16:20.41] - Joseph Thacker
Yeah.

[00:16:21.41] - Justin Gardner
So that is definitely something that is going to be— is going to be widely applicable, I think, with AI agents. And maybe we were just sleeping on as humans as well. I don't know.

[00:16:31.20] - Joseph Thacker
Yeah, I wonder if it's like sometimes hard to recognize or notice, at least for me personally, I don't always like— I think that basically Incoming headers, like response headers, are something that I did not look at enough pre-AI. And so like when some random API request that I was testing in Repeater, uh, or in Replay or whatever would, would have a set-cookie header, I don't know if I always noticed it, you know, or like, yeah, because, and then it's also kind of annoying to like go set that up because I feel like we don't have like a Replay-based cookie jar or whatever where like then it would automatically use it or whatever.

[00:17:04.30] - Justin Gardner
Yeah, yeah. So I don't know. I think that's really interesting because it comes to a point that we've talked about a little bit more on the, uh, on the pod in the past. But, um, I think cookies are a little bit of a, um, under— underappreciated component right now in, in web security. Like, I think if I— I, I can't even think of a target in which case where like actually navigating through the app and looking at all of the cookies that have, that are attached to my request, that I know what every single one of them does.

[00:17:38.75] - Joseph Thacker
Right.

[00:17:39.05] - Justin Gardner
Right. Like, like a huge black box, very intimate with these applications. Right. But I still can't tell you what every one of them does. Cause there's so many like stupid analytics garbage, you know, like that gets, that gets tied in there. So I think that, um, some automation or some, um, tooling surrounding Specifically auditing all of the cookies on all of the endpoints and what they do and what, what are necessary, what, you know, what changes the response.

[00:18:05.14] - Joseph Thacker
Notice the diffs, like, cause each of those represent some sort of code path downstream.

[00:18:09.53] - Justin Gardner
Yeah. And I definitely have seen in the past as well, some super big brain, single target, you know, home, home target people, right. Where they just hack on one program. I've seen them find these sort of vulnerabilities before where they're like, That's a weird cookie. I haven't seen that cookie. That tells me something, right? You know, and they're like, okay, I need to go hack on that. But I think we just don't— because there's so much spam in there, we don't have as much of a radar for that.

[00:18:39.96] - Joseph Thacker
Yeah, just glaze over.

[00:18:42.30] - Justin Gardner
Yeah. Yeah. Very interesting. All right. So since we're on a hackbot stuff, I have been specifically prompting my AI to look at GraphQL lately. And I think that that is a really, really helpful area.

[00:19:02.25] - Joseph Thacker
Yeah.

[00:19:02.90] - Justin Gardner
Because they can have different middleware and auth schemes, you know, for different queries and mutations and stuff like that, which is something that I've seen. So having it, you know, literally hit every one is, is really valuable. And then it can also— one of the things I've noticed is that my AI does a really good job of telling me whether the denial is happening at like a middleware level or if there's something like beyond that. Uh, I'm trying to remember the exact term that it used. I've got a bit of a headache right now, so it's not coming to me, but there's like an authorization layer that it is calling out on a regular basis that's normally built into these Apollo servers. Apollo-based GraphQL servers that it can tell. It's like, oh, okay. It's not at this layer. It's at this layer. So that tells me that we need to dig a little deeper here, right?

[00:19:50.46] - Joseph Thacker
I will say that's the most common issue that I've seen for false positives too, though, because a lot of times it'll find a way to get past the middleware auth.

[00:19:59.41] - Justin Gardner
Yeah.

[00:19:59.80] - Joseph Thacker
And it'll be like, oh, instead of a 403 here, I got a 401. And so I know I'm past that first level auth. So this, uh, you know, null byte that I put at the end of the API call is actually a bug, and it'll report it to me. Like, it'll like— it'll like give me and JD a report, and then I'm like, You had no impact with that.

[00:20:16.50] - Justin Gardner
And how does that get past your validator, bro?

[00:20:19.36] - Joseph Thacker
Well, because it is finding some sort of like, like the validator is like, yeah, it is legitimately bypassing that middleware auth. So we need to report this because there is a—

[00:20:27.89] - Justin Gardner
because it actually is valid.

[00:20:29.19] - Joseph Thacker
It just doesn't have impact. Right. And so I need to make the validator have more of an impact-focused awareness. But it really is like there's some vulnerability in their nginx config or in something that does allow it past one of those auth gates. But if it can't get past the other one to actually access the data, then it's not impactful enough for a bug bounty report.

[00:20:46.26] - Justin Gardner
Right. So anyways, right. I feel like my validator's kind of a hard ass. I feel like I need to tell him to chill out a little bit. Cause like, like my, my signal of stuff that gets through is very high, uh, probably 80 or 90% plus. But, uh, I know that I'm missing stuff.

[00:21:04.10] - Joseph Thacker
Yeah.

[00:21:04.52] - Justin Gardner
I know that the validator's being like, nah, that's not it. And also I know right now as well, is one thing I've got to tweak in mine is that the agent will. Routinely find escalations and I don't have—

[00:21:17.04] - Joseph Thacker
oh, you need escalation agent bad.

[00:21:18.64] - Justin Gardner
I know, I don't, I don't have an escalation workflow for it.

[00:21:21.69] - Joseph Thacker
Yeah, you need that.

[00:21:22.22] - Justin Gardner
So it's like finds an escalation, tries to submit another bug report, the validator's like, nah, that's a dupe. And then, and now I don't—

[00:21:28.10] - Joseph Thacker
you lost the escalation.

[00:21:29.17] - Justin Gardner
Yeah, I'm like, dang it, you do have to fix that. Yeah, I need to fix that. So, but going back to the GraphQL thing, I think, um, I think that for me, I've seen across 3 different targets that I've put it up against right now that the auth layer thing that it's doing is accurate that I've seen. And, you know, surely it freaks out about stuff that I, you know, that it shouldn't, you know. But I trust its intuition a little bit there. And I've had GraphQL reports on all 3 of those targets that were valid. So, just using anecdotal evidence. That's legit.

[00:22:06.36] - Joseph Thacker
No, it's great evidence.

[00:22:07.27] - Justin Gardner
Yeah.

[00:22:07.52] - Joseph Thacker
I mean, I think that's— Uh, it's something that's pretty annoying to test as a human, but the agents are pretty quick and good at hacking.

[00:22:14.80] - Justin Gardner
Yeah.

[00:22:15.45] - Joseph Thacker
Um, I wanted to mention this. I know I mentioned Fable a minute ago, but I wanted to tell you, um, and kind of shout out somebody, you know, Pom. Um, he has a, uh, a penchant for building stuff on the side. He doesn't share it very much, but, um, when Fable dropped yesterday, he messaged me, was like, bro, this thing one-shot this. And it was an entire like, I won't get into the spoiler details in case he doesn't want me to share, but just imagine an entire video game with like a— because like all the stuff he's built before this has not been video game related, so he wanted to do something different. Fable built an end-to-end game that was freaking cool, and it would completely one-shot it end to end with like graphics and animations and all the things. So anyways, I just want to mention that because if people are—

[00:22:57.77] - Justin Gardner
how much did it cost him, do you know?

[00:22:59.88] - Joseph Thacker
Well, I think he just uses his Claude Mac sub.

[00:23:02.10] - Justin Gardner
Yeah, you think so? It was, it was within CloudMax, you think?

[00:23:06.06] - Joseph Thacker
Uh, it specifically, he said it was within his Cursor subscription, but he also, but he also has CloudMax subscriptions, but they both have Fable right now. And anyways, the reason I want to mention this— oh, sorry, it goes away July 7th. So when listeners are listening to this, they're not gonna be able to tap into it. I'll tell you, Justin, Fable is only included with your Cloud subscription until July 7th. After that, it'll be API only. So you should use it for building anything like non-hacking related where you'll get a bunch of rejections in the next 5 days.

[00:23:31.95] - Justin Gardner
It's worth prioritizing it. It really is a step up.

[00:23:34.00] - Joseph Thacker
Um, you know, I, um, I follow this guy named Taylan on X and then I also am good friends with Hrishikesh who we've talked about before. They both have hard workflows that even Opus 4.8 like fails at or struggles at or can't do well at. And for both of them, it's made like huge leaps. So I think that it is worth using it. Yeah.

[00:23:51.68] - Justin Gardner
Wow. Okay. Yeah. I've got some dev projects I'm going to have to do on the side here within the next couple of weeks. So maybe I'll just try to front load those before the 7th and, uh, and get, knock those out.

[00:24:03.14] - Joseph Thacker
Yeah.

[00:24:03.31] - Justin Gardner
Sorry. Sorry for people that are listening to it. Uh, it's too late for you. Uh, yeah. Going, going back to the AI bots though, the last thing that I had on my list to talk about there was, um, I've, I, I don't know. I feel like this is kind of an obvious one. So, you know, it is what it is, but I've been telling it, be very like your number one job. Is to get the JS files. You know, get the JS files, get the JS maps, get the lazy loaded stuff. That is your number one mission. And then build out every single API endpoint. And I took inspiration from that episode with Birdcat. And I've implemented a sort of a coverage system where I'll tell the AI, you know, your goal is to grab the JS files, grab all the API endpoints, and then systematically push them into this coverage tracker.

[00:25:00.74] - Joseph Thacker
Yep.

[00:25:00.99] - Justin Gardner
Right? And then pop off that queue and just start churning through them. Right? And, uh, so I had— I was running it on a target for a while. And, you know, it was churning out some bugs for a while. And then it started hitting sort of dead end, right? And then I implemented this feature and it got another wave of bugs.

[00:25:20.25] - Joseph Thacker
That's awesome.

[00:25:20.98] - Justin Gardner
Uh, from this. So even if you don't go full send brute cat mode where you're like actively checking that it hits every endpoint, you know, checking Kaido, you know, correlating all these requests, implementing auth at that layer, even just giving it a way to track attack vectors and keep itself accountable for whether it's tried all of the API endpoints or whether it's like, oh, auth is here. GG, you know, then I think you will get a good amount of output out of that. Have you done anything with like coverage tracking like that at all?

[00:25:56.50] - Joseph Thacker
Or, um, yeah, like a little bit. So it's mostly on— well, there's kind of two things. One is I definitely have it do like basically a recon map where with JD's like JS stuff, and then, and then have it, uh, crawl the app and then combine all of that into like a treasure map of sorts. And then, and then have the kind of like boss agent delegate, um, like hacker or hunter agents to then like make sure it hits all the scope. But I've still been relying on that boss agent to like, you know, and I tell it to be really tough and make sure they actually like test all the things. But I think having some sort of system, systematic, or like, you know, actual functional way to force it like you, like you just did, is probably smarter.

[00:26:40.53] - Justin Gardner
Yeah, yeah, good, good results out of it. And, and I think also prompting it directly with things that you've noticed about the architectural flow of these organizations as well is, is a big thing. And this kind of goes back into the human in the loop sort of situation where even if it, it takes, you know, let's say it finds one vulnerability and you say, okay, well, clearly they've got a JWT validation issue here, or like this is an auth problem that they've got across the board, right? Telling it, okay, now double down into that, right? And go and find it. One of the ones that popped up recently for me was Cognito related stuff. So this one org I was hacking on used Cognito pretty heavily. And it's just Cognito is fairly easy to, to misconfigure. I'll put a link in the show notes, but there was like a, a talk hunting for AWS Cognito security misconfigurations by Yassin back in 20— 22 at NahamCon. Um, and dude, this stuff, I literally have found this week with that on a, on a public program.

[00:27:46.64] - Joseph Thacker
Wow.

[00:27:48.07] - Justin Gardner
You know, 5-figure bounties, you know, crit, uh, with this. So they're still there and Cognito is used fairly often. The one that I found was security misconfiguration number 3, which is, uh, privilege escalation through writable user attributes. So essentially there's like custom attributes that are on your Cognito account. That define maybe like your role, your permissions and stuff like that. And, um, those are like default writable if you don't want them as unwritable. So you just log, even if it gives you a bearer, right?

[00:28:19.51] - Joseph Thacker
Right.

[00:28:20.16] - Justin Gardner
And you never hit Cognito, find the Cognito pool or whatever, uh, hit it directly and then modify those attributes. And now you're super admin.

[00:28:28.84] - Joseph Thacker
That's insane. Um, yeah, I didn't, I like, I never thought about like how there are basically third-party auth slash permission services that can be attacked more at scale.

[00:28:40.96] - Justin Gardner
Yeah, yeah, I think Firebase has a lot of this stuff too. So, so especially as we're getting more Vibe Code stuff in prod, this stuff is going to be a little bit duct taped, you know, and, uh, if it starts holding user data, then let's go, you know.

[00:28:58.52] - Joseph Thacker
Yeah, and they are often very hard to configure and The agents aren't always configuring them. It might be like the user spun up a— oh man, what's the name of that really common vulnerable thing? We had the .equals operator. It's like the really common row-level security.

[00:29:14.16] - Justin Gardner
.equals operator. Yeah, you talked about this. What was it? Supabase?

[00:29:17.33] - Joseph Thacker
Supabase. Yeah, I feel like everything these days uses Supabase. And oftentimes it's created via a user in the— actually, you and I did this on Firebase for Shift back in the day. It's so easy to just misconfigure that.

[00:29:31.18] - Justin Gardner
Shh, shh, don't say— don't call us out on that, Rezo. It's perfectly secure.

[00:29:36.74] - Joseph Thacker
Yeah, it's gone now. I deleted it.

[00:29:39.06] - Justin Gardner
I was like, what are you doing? Yeah, it is what it is.

[00:29:46.45] - Joseph Thacker
Yeah, I want to talk quickly, like, I know we're running low on time. We only have like 8 minutes left, guys. But yeah, kind of 2 quick MCP things. One, X just released their MCP, so probably a good target for people to go try to test if they want to. Um, it's just, uh, well, you can just Google it. It's XMCP, but, uh, and we'll have the link in the show notes. The other thing is I have been mapping out Google's MCP infrastructure and they have like 700 and something MCP servers.

[00:30:14.53] - Justin Gardner
Holy moly.

[00:30:15.74] - Joseph Thacker
Yeah. And a lot of them, uh, like if you just do tools list on them, they just list all their tools. And, um, wow. And then there's a lot of hidden tools as well that are not advertised, which I think is like a breach of the MCP, uh, protocol because if you do a list and you're like, oh, this thing can do this one thing, yeah, I'll approve it. And then actually under the hood it has a bunch of other tools that it can call. It feels very— I don't know, if they accepted that as a bug, then they would have to pay out like $100 to me. But anyways, bugs, not bucks.

[00:30:45.89] - Justin Gardner
Um, but they'd have to pay me $100. Yeah, that's too real, bro. It's too real.

[00:30:52.34] - Joseph Thacker
But yeah, I don't know, it just feels funny and weird because I feel like, uh, I feel like the industry has kind of like moved on from MCP, and that's clearly not true. It's still like a huge protocol and a huge standard, but it's just surprising to me that Google has 700+ MCP servers.

[00:31:04.60] - Justin Gardner
Am I misunderstanding that MCP has to dump the results directly back into context? Like that, that's the, that's my biggest thing with MCP is like, why, like when you're dealing with something that's going to be dumping data back into the LLM, you should allow the LLM to more precisely query exactly what it wants. And unless you build that directly into the MCP tool. Like, this is why Skills are ridiculously overpowered because then you can just run it through jq and grep, right?

[00:31:33.39] - Joseph Thacker
Right, right. Yeah, because I mean, one, you can have whatever—

[00:31:36.05] - Justin Gardner
it's a no-brainer.

[00:31:36.99] - Joseph Thacker
You can have whatever CLI interface you want and it can be completely malleable to any kind of API path or endpoint. Like, it can be GraphQL under the hood, it can be anything with Skills.

[00:31:45.50] - Justin Gardner
Maybe model providers are pushing this because they want more token usage. I don't know.

[00:31:50.41] - Joseph Thacker
No, I think that the biggest benefit which you don't get from Skills is the auto-updating nature and auto-broadcasting nature of what it supports. So if you as a provider want to update what the server can do and the customer is still using the old skill, there's not a way to force update the users. MCP is ideal for enterprises that want to be like updating their stuff because before the call is made, it basically has to ask the server, what do you have? And then it advertises it back. I actually have built a few custom MCP skills for clients. Like just like local businesses. And this is actually kind of a pro tip for anyone out there who's doing that. Um, and you actually could potentially use it in a hackbot in a really cool way. Basically, uh, wow, now I want to rebuild my hackbot to use skills like this. Uh, they're almost like, you can almost call them like beaconing or whatever. You have to really, really trust them. But the, um, what you do is you say in your skill, hey, this skill is used for this niche, this service, this, uh, uh, the thing I want you to use it like all the basic stuff you have in your skill. But when it comes to actually the usage of it, You say like, the first thing I need you to do before you use the skill is hit this endpoint to see what functions it has. And so, you know, like, and so then what you can do as the MC or as the skill maintainer, you can now automatically update your list of functions that can be called through this skill. And then it advertises it and it broadcasts it. So then when the agent asks, hey, what's available now at the start of this like new run or new turn, it says, oh, you can do these 10 things.

[00:33:27.01] - Justin Gardner
Nice.

[00:33:27.41] - Joseph Thacker
And so then agent—

[00:33:27.85] - Justin Gardner
Little combination.

[00:33:28.75] - Joseph Thacker
Huh?

[00:33:29.80] - Justin Gardner
A combination of both MCP and Skill.

[00:33:32.03] - Joseph Thacker
Yeah, yeah, it is. Yeah, that's a great point.

[00:33:34.20] - Justin Gardner
Yeah.

[00:33:34.36] - Joseph Thacker
And so anyways, I found that very useful because if a customer has deployed my Skill to like 10 of their endpoints, like they're like secretary and they're dev, and then they want me to add a feature to it. Now they're going to have to go redeploy and deploying it like little SMBs is really hard because like, how do you deploy a Skill? Well, you can go download a zip file and then because they're using the desktop app, you drag it in. And so I just want them to be able to install the Skill one time and then I can update it from my server. And so that's good.

[00:34:00.91] - Justin Gardner
Yeah, I like that, dude. That's a good, that's a good, good move there. I think there's probably a nice happy medium between the extensibility and updateability of the MCP and the token efficiency chainability of the, of the skill of the like command line based skill. Yeah. All right. Yeah. Let's see. Do you want to, you want to talk about the US South event really quick before I hit some couple, couple things on AI Sandbox escapes?

[00:34:29.92] - Joseph Thacker
Yeah, sure. Um, yeah, we've got just a couple minutes here left. Um, there is a US South, a HackerOne Ambassador US South, um, like kind of event that is going to happen the second half of July. And there's a link in the show notes if you want to go to— if you want to attend it. Is it going to be remote only? I think remote only. Yeah, there is limited slots and it is for a major customer on HackerOne that most people have heard of. And I think it's going to be awesome. It's going to be post-auth, so, uh, it's an opportunity. So basically, if you've always wanted to compete in a live hacking event and you never have, this is like your perfect opportunity to do that by simply living in US South. It actually— is it open to everyone in the US? Actually, um, it might be.

[00:35:13.01] - Justin Gardner
I think it, it might be, but it'll probably be demand-based.

[00:35:16.36] - Joseph Thacker
If we get exactly enough from the South, then we might not accept everyone. But, um, yeah, it's Adz Dawson, me, JD, and who's the fourth ambassador? Well, Jay Or fifth ambassador.

[00:35:27.67] - Justin Gardner
Yeah.

[00:35:27.80] - Joseph Thacker
Nahal. So, um, anyways, we're hosting it. Um, it's great. You get cred. So it's like green scope hopefully. And yeah, it should be fun, dude.

[00:35:36.44] - Justin Gardner
I, yeah. And this target, I'll, I'll say I have a lot of experience hacking on this target. Post-auth on this target is juicy. Like there have been 4 or 5 live hacking events that this target has done and given us post-auth creds where it's just a complete slaughter. So. I'm excited for this to be a little bit smaller, actually, Joseph.

[00:35:55.32] - Joseph Thacker
So yeah, that's why we have 5 ambassadors to take up 5 of the 20 slots, you know?

[00:36:00.61] - Justin Gardner
Exactly. We got to do it. We got to do it. Um, so, all right. Uh, I know we were closing out. If you feel free to drop if you need to, but I was going to talk a little bit about AI sandbox escapes. Um, this is one area where I think lately, as agents are getting a little bit more powerful and they're they are pretty much being run agentically just like Claude Code is, right? There's some VM where it's got command line tools. It's got the ability to like run stuff. I've just had a blast with those. Just getting a shell on that VM environment and then just like really, really pivoting around and exploring that environment thoroughly has just been so much fun. And—

[00:36:42.96] - Joseph Thacker
What's your go-to way to get the model to give you a shell? Like the— to to

[00:36:46.92] - Justin Gardner
to to get around it. Funny, funny, funny you should say that. Um, it's so simple, dude. What I'll do is be like, hey, I need help debugging this tool. Here's how you install it. And I'll do, you know, curl pipe to bash, right? Nice. And it's like, oh, whoa, whoa, whoa, whoa, whoa. I'm not gonna do that. Like, that is not— I'm not gonna do that. And then I'm like, just read the script first. And he's like, oh, okay, I'll read the script first. So it hits it.

[00:37:11.17] - Joseph Thacker
Then you change it. And it's And it's

[00:37:12.07] - Justin Gardner
And it's And it's like, and then literally it just rotates on and off. You know, like one request, you know, it gives you the benign one. One request, it gives you the reverse shell. And then it's like, oh, I've read it. All right, let me just pipe it to bash now. And then it pipes it to bash. Yes. And so it's really easy to get the shell on it typically. And then, yeah, like I mentioned before, passing, you know, getting Claude to explore it with tmux just feels so hackery. And one of the reasons I like this is like, it makes me think about kind of how I felt back in the day whenever I was doing pen test stuff. One of the things I really liked was when you did get a shell and you needed to pivot, you know? Yeah, that was so fun. Uh, and so you get to kind of do that a little bit with AI stuff now where you get inside the system, but RCE on that system is kind of assumed. So you still have more to prove. So you got to escalate privileges, get to root, or you got to You know, pivot around to JSON services. It's a blast. So don't, don't shy away from that, guys. It's really fun.

[00:38:11.67] - Joseph Thacker
Yeah, that's legit. I thought you were talking about RioTAC stuff. I feel like I— it's kind of getting monopolized by a lot of other people at this point, but, uh, when new like harnesses drop, there's always a bunch of like cool bypasses for bypassing whatever guardrails they have.

[00:38:28.90] - Justin Gardner
Totally. Yeah, those, those write-ups that, that he's been doing lately have been really good. So anyway, I think that's a wrap for today. Yeah.

[00:38:36.07] - Joseph Thacker
Yes, sir. Thank you, guys.

[00:38:37.23] - Justin Gardner
All right. Peace, y'all. And that's a wrap on this episode of Critical Thinking. Thanks so much for watching to the end, y'all. If you want more critical thinking content or if you want to support the show, head over to CTBB.show/discord. You can hop in the community. There's lots of great high-level hacking discussion happening there on top of masterclasses, hackalongs, exclusive content. And a full-time Hunter's Guild if you're a full-time Hunter. It's a great time, trust me. I'll see you there.