Skip to content

Conversation

@benleoau
Copy link
Contributor

No description provided.

@theRealKLH
Copy link
Collaborator

theRealKLH commented Aug 14, 2025

@benleoau I'd request that you not submit PRs to update "aggregator" scrapers unless it's to fix already included studios. This is why the "custom" scrapers option was created. Each person picks exactly which studios they want to scrape. Every time we change what's being scraped GLOBALLY from aggregate sites, we run the risk of complicating someone's existing setup (duplicating/overwriting/deleting/blah blah).

If you want to share snippets of your "scrapers.json" for others to use as custom sites, I'd suggest you jump over and join us on discord.

case in point, I'm missing previously scraped studios that were removed from SLR that I had downloaded & matched scenes for but they were removed from my library when someone removed the site from the scrapers.json

@arancour69
Copy link

good progression 4 updates to content

@arancour69
Copy link

100%

{
"url": "https://vrporn.com/studio/vrmassage/",
"name": "VR Massage",
"company": "VR Massage",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

@benleoau
Copy link
Contributor Author

benleoau commented Aug 14, 2025

I have added aggregator because there is no other options, none of them has the website, they are the only scraper that we can have. they are small studios that mostly does not have even website.

@benleoau
Copy link
Contributor Author

HoloGirlsVR website has gone, we have to use aggregator, vrporn or slr. it mean even if you have HoloGirlsVR in your inctance, it is not working anymore. please have a look to their website https://www.hologirlsvr.com/ it is shut down

@arancour69
Copy link

benleoau:feature/addNewScrapers

can you send me a windows version 4 me to test?

@benleoau
Copy link
Contributor Author

vrplayful: website is a redirect to aggregators
bs-production: no official website
lethalhardcorevr: it has website but there is no scraper for it in codebase
vrteenrs: no official website
allanalvr: no official website
vrhard: no official website
vrmassaged: no official website
hologirlsvr: official website shutdown, I have removed the custom scraper
vrmassage: no official website

@benleoau
Copy link
Contributor Author

benleoau commented Aug 14, 2025

btw: I have found a bug in virtualrealporn.com scaper, it cannot find the image url and try to record whole json in the database which caused a crash in the app. I cannot put brackpoint in windows so I cannot debug it. this is my first time I am developing Go. #2049

@benleoau
Copy link
Contributor Author

benleoau:feature/addNewScrapers

can you send me a windows version 4 me to test?
https://we.tl/t-K3T4vcLghZ

@theRealKLH
Copy link
Collaborator

I have added aggregator because there is no other options, none of them has the website, they are the only scraper that we can have. they are small studios that mostly does not have even website.

I understand why you did, but you should just offer this as "custom scraper" snippets for users to add themselves. You are just "cluttering up" the scrapers list with studios from an aggregator that not everyone needs.

@benleoau
Copy link
Contributor Author

I have added aggregator because there is no other options, none of them has the website, they are the only scraper that we can have. they are small studios that mostly does not have even website.

I understand why you did, but you should just offer this as "custom scraper" snippets for users to add themselves. You are just "cluttering up" the scrapers list with studios from an aggregator that not everyone needs.

I am totally confused, I can see lots of scarpers from VRPorn and SLR in codebase. I thought it is a normal approach to add more. is there any document how users can add custom scraper? I have edited my local scrapers.json but when I run the app, that file is going to over-write by the original one.

@arancour69
Copy link

no worries

@benleoau
Copy link
Contributor Author

benleoau commented Aug 14, 2025

I think I have found it, but how I can set the source as VRPorn? It seems Main Site is different.
image

@theRealKLH
Copy link
Collaborator

I have added aggregator because there is no other options, none of them has the website, they are the only scraper that we can have. they are small studios that mostly does not have even website.

I understand why you did, but you should just offer this as "custom scraper" snippets for users to add themselves. You are just "cluttering up" the scrapers list with studios from an aggregator that not everyone needs.

I am totally confused, I can see lots of scarpers from VRPorn and SLR in codebase. I thought it is a normal approach to add more. is there any document how users can add custom scraper? I have edited my local scrapers.json but when I run the app, that file is going to over-write by the original one.

Don't be confused. Grin...

The reason for all the existing studios was before everyone had the ability to do their own studios via the CUSTOM scrapers. Now it doesn't make since to update "official" studios on aggregators cuz you can add whichever ones YOU want without affecting anyone else's installation.

@theRealKLH
Copy link
Collaborator

theRealKLH commented Aug 21, 2025

I am going to say... Close this PR and open a new one with JUST the logo updates. Aggregators "studio" code will probably not get merged into release.

Please read this PR on the configurable scrapers list. This negates the need for what you are trying to accomplish...

#1113

Either add studios via the GUI (advanced/custom sites) or add them to the proper place in the scrapers.jason file.

@benleoau benleoau changed the title add a few new scrapers of vr porn and fix all missing logos fix all missing logos Aug 23, 2025
@benleoau
Copy link
Contributor Author

@theRealKLH
Do you think we need to remove www.hologirlsvr.com scaper?

@crwxaj crwxaj merged commit 33dc5b1 into xbapps:master Oct 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants