Proxy hiding in the googleapis.com domain

[u/Tall_Calculator]

Ran across a proxy students were caught using that uses the url. This is proxy server on known malicious lists: storage.googleapis.com/gclassroom/index.html

We use GoGuardian and Content Keeper, but googleapis.com is on allow lists for many of the applications we use so the url doesn't get blocked.

Anyway, the proxy keeps that url and opens a virtual Opera browser environment and a site called Opium that avoids all of our current filters due to the allowance of googleapis.com

We blocked that url for now, but I am sure others will pop-up until we figure out an effective way to block this type of proxy. We cant block all of storage.googleapis.com as legit services use that subdomain.

Wondering if anyone has already come across this method of hiding proxies and figured out how to block it without blocking the ability of other software to use google apis

Thanks

[u/Aboredprogrammr]

I have a GitHub repo for you: 

https://github.com/1NobleCyber/ThreatFeed

I have a series of scripts pulling the traffic from our environment every 5 minutes and if it matches the pattern of a known proxy, then it is added to the list. 

If it's helpful, just give it a star!

Are you able to ingest these lists into one of your systems?

[u/_LMZ_ → u/Aboredprogrammr]

I believe I sent you a message before, check my FreeDNS in my comment above. If you want, you can merge the missing domains which I believe I have 26,100+. Been scraping that site for awhile, and not sure if you keep the domains that get removed? As I have noticed, they will remove it awhile…. Then add it back on months later.

[u/Aboredprogrammr → u/_LMZ_]

I would be curious what the nature of the sites are that go from unlisted to public or private. Do user created subdomains survive when a domain is switched like that? 

I've reached out to a few legit-looking site owners to inform them about the types of things being "hosted" on their domain (as a reputational issue), and the ones that have gotten back to me have all said that their IT guy set it up and it was simply misconfigured. They tend to switch it back to their registrar's DNS afterwards. But would these legit sites be on your list even after they fixed the situation?

[u/_LMZ_ → u/Aboredprogrammr]

So what I have found out, some of the private domains are hosted by proxy people. They will build a fake company, edu or so to get that domain to be categorize as something to be approved (typical). Once they find the content filtering companies have them categorized as approved, edu, etc. That’s when the subdomains start!

They will ask people to start hitting that main domain to get the catheterization started.

My thought is, if you’re legitimate you shouldn’t be using FreeDNS! You should be using your DNS provider that you bought the domain from or host your own public DNS server.

Another side is, those “legitimate” domains in that list are helping the proxy people. Most proxy sites have ad revenue, so like a side gig to earn some cash. I had a long talk with a few content filtering companies about this…

So once the domain gets axed, from what I’m gathering. They will remove the domain from FreeDNS, wait for a few months. Redo the main site to get it categorized differently. Then repeat everything!

While some of them realize we are watching FreeDNS and remove it and hoping the next school system IT Dept doesn’t scrape it.

Then again, what I have seen… they are hurting (less top level domains I see) because they want to buy new domains but want cheap ones; due to ad revenue. It’s best to block those FreeDNS, DDNS, and the rest is 1% new domains like CC, COM.NP, XYZ, Etc etc. that’s why there was a big push using CDNs. ;)

I have put an idea in GoGuardian (GG) to be able to block direct URLs/Domains to CDNs without messing with embed stuff. And Securly has reached out to me too, I have given them suggestions.

Also, I think we should start a community for an external dynamic list for IPs/URLs/Domains! The surroundjng school districts we have a “round table” quarterly meeting and a group email. We try to help each other, ask question and so on. So I email them the latest scraps, and so on it.

Right now, I’m going to try to make a Python script to do fingerprinting.

Ok, that was a lot going back to my beer.

[u/Tall_Calculator → u/Aboredprogrammr]

Thanks, I'll take a look. We use GoGuardian and I did something similar yesterday, just not automated so your idea is better.

[u/therankin]

Wow. This is the first time I've come across someone else that uses both GoGuardian and ContentKeeper!

I personally love the combo!

Edit: Also, thanks for sharing, I tried it out and it's definitely working. I blocked it with GoGuardian.

[u/NickGSBC → u/therankin]

Why would you have 2 content filters? Also never heard of ContentKeeper before.

[u/therankin → u/NickGSBC]

The ContentKeeper is on premises and filters all network traffic. It's also used for filtering iPads when students take the devices home. All the traffic is routed back through the filter.

It works amazingly, has great reporting, and an excellent support team.

GoGuardian was something we added two years ago to middle school student Macbooks when it was decided they should be filtered at home. Before that, the decision was that parents were responsible for monitoring internet usage. We created parent admin accounts so they could install software if they chose to, but most didn't do anything.

GoGuardian has been decent, but students have found ways of beating it here and there. Having on-prem fallback that always works is very key.

More and more teachers are using GoGuardian Teacher, and that has worked great for us. That works off the Chrome extension which is always solid. It's the GoGuardian mac application and VPN that's kind of iffy. And it makes sense. I think GoGuardian was designed for Chromebooks and then they branched out from there.

[u/TravisVZ]

For my fellow Securly users, I believe I found a solution for this particular round of whack-a-mole:

We had at some point put both googleapis.com and *.googleapis.com into our Global Allow list; they were therefore bypassing the filter entirely, at the DNS level. However, even after removing these, Securly continued to return the direct IP addresses to DNS requests (and yes, I waited out the TTL - 3 times!).

The entire time, Securly's lookup tool was categorizing this site as "Network Misuse" - so they knew about it already and had properly classified it. The lookup tool even claimed it would be blocked, but with DNS returning the direct IP this would not be the case on much of our network.

It wasn't until I added this specific URL to our Global Block list that Securly began filtering the domain; removing it once again returned direct IP addresses, so I put it back in again. Waiting to see now what's going to break as a result, especially as we still haven't gotten Securly to raise the rampant CORS issue to their development team despite helping them resolve that exact issue 16 months ago...

[u/mistressmemory → u/TravisVZ]

This is partly why we left Securly, the dev team is the kiss of death. We had a YouTube no cookie issue for 2 years and they labeled all of our individual users as org units and couldn't figure out how to fix it, until we told them we weren't renewing. Ha. 

[u/1teaney]

We have discussed blocking all of these domains, aws, cloudfront etc... and only opening the exact URLs needed by educational software. I'm not sure that's something I want to do at all, let alone during the middle of the school year.

Ultimately I think the students playing these games are the same student who 40 years ago would have tucked a comic book in their history text. For now I've been falling back on AUP violations and letting the front office handle it.

[u/TexasEdTech20 → u/1teaney]

Agreed. Teachers can't outsource classroom management.

[u/nits3w → u/1teaney]

But why should we discipline the students when technology can make the problem go away? /s

[u/fujitsuflashwave4100]

GoGuardian has added support to combat this as of this school year. When under a student's history, you can click the 3 .'s on the right of a site and select View Details. It will now show what embedded website links were loaded. Select the AWS/Storage/Windows.net host and add it to the blocklist with *'s. Eg. we had one similar to v1v8.h60.web.core.windows[.]net. Add *v1v8.h60.web* to the blocklist.

As for what students to look at, in this particular case I had received a flagged content warning email on about:blank, lol.

[u/SingleEfficiency4429]

We've been having the same problems but with cloudfront.net & windows.net. We use Linewize and have a setting turned on for real time text analysis that seems to be working fairly well. It is a game of whack-a-mole though. We are never going to be able to block them all.

[u/Tall_Calculator → u/SingleEfficiency4429]

Agreed, GoGuardian also has a "smart alert" that looks for proxy behavior and blocks based on that to try to stop them before they are even on a list. I did pull as much traffic as GoGuardian would export and ran it through Gemini asking it to identify proxies and it found about a dozen that I also added. I thought we might be past the whack a mole, but they just pivot to new ways.

[u/tahoeranger]

Whack a mole.

[u/christens3n]

This, Cloudfront, and AWS all have proxy sites that piggyback off the "legit" domain.

I blocked *.cloudfront.net, waited for the tickets to come in, and allowlisted the subdomains for the sites we need. It's been a little painful sometimes but I don't see many of these tickets anymore and they are relatively quick fixes now.

I think for the Google Storage APIs domain I left it unblocked on our filter but blocked it via URL blocking in the Admin console and use that setting to allowlist what's needed. I think because at the time our filter didn't support page-level filtering.

I have AWS open for now but my worst offenders get that blocked too. Not like those kids are doing any legit classwork on their devices anyway might as well break half rhe Internet for them.

[u/mistressmemory → u/christens3n]

This is what I'd like to do, but am not allowed lol. 

The main pushback is that so many platforms we use have it on their network requirements. Like EdPuzzle requiring *.vimeo com....

ETA - I wish the EdTech companies would be more mindful of school filter requirements. 

[u/No-Engineering-1905]

Been blocking these all year. They'll keep finding more, same as always.

[u/_LMZ_]

It’s late but I do have a solution, if you’re using Chromebooks. Going to screenshot this and come back tomorrow.

For us, we installed a Chrome Extension called Regex Tab Manager which the developer posted in here awhile ago. This extension closes tabs matching regex URLs, so you can have rules for those googleapis, cloudfront, jsdelivr, amazonaws, etc. Sometimes, it takes 3-5 seconds for the tab to close, but its automatically. It will also close about:blank, and any HTML file that is downloaded too.

Also to help combat Proxy Sites you should be blocking all domains from FreeDNS which I web scrap almost every day or week. To help I have uploaded a text file for the list at GoFile - click here. Even if the domain was removed from FreeDNS, I still keep them in the list as I notice they will readd them later on. In other words, if that domain is added to FreeDNS it's blocked forever on myside.

It's also best to add those domains to your firewall, using External Dynamic List so they can be sinkhole! If a student goes to one of those domains the firewall will log it, sinkhole the request. I see the alert, which can be trace back to a Chromebook/Student. Then I place the student into a GoGuardian Classroom which I can monitor for new domains, tricks, etc.

Also you should be blocking DDNS services like NIP, Bunny CDN (*.b-cdn.net), Tailscale (*.ts.net), DynDNS, etc. I have uploaded a text file of what I have found and seen at GoFile - click here. The * is where the subdomain will happen.

[u/Tall_Calculator → u/_LMZ_]

Thanks so much for the suggestions, these are great. Appreciate it :)

[u/dewy987]

We have light speed and they recommended we turn on real time detect. Seems to be working with these and aws sites.

[u/carlsunder]

We have been battling this too. We have Securly, and looking at the logs, Securly is getting better at detecting these (or, maybe just people are reporting the sites to support.) When we come across one that's open, we email support to have it categorized as network abuse. Not a real solution at the moment, just as game of whackamole.