It is currently April 19th, 2024, 2:39 pm



Post new topic Reply to topic  [ 10 posts ] 
Author Message
 Post subject: Regex Search For Duplicates
PostPosted: May 19th, 2023, 2:19 pm 

Joined: November 23rd, 2014, 6:31 am
Posts: 276
Hi,

I am wondering if there is anyway to use search to find duplicate posts by certain string in Subject title.

Example:

Code:
|40:54| MySecretLifePOV FREE Page - 2022.09.08 - @mysecretlifepov @katalinakyle ID: 2584453353
|40:54| MySecretLifePOV FREE Page - 2022.09.08 - @mysecretlifepov @katalinakyleu00a0and ID: 2584453353


Code:
|41:15| MySecretLifePOV FREE Page - 2022.11.14 - @mysecretlifepov @mollymaeroyvip @badbellaxo ID: 2669179277
|41:15| MySecretLifePOV FREE Page - 2022.11.14 - @mysecretlifepov @mollymaeroyvip @badbellaxou00a0both ID: 2669179277
|41:15| MySecretLifePOV FREE Page - 2022.11.14 - @mysecretlifepov @mollymaeroyvipu00a0came @badbellaxou00a0both ID: 2669179277


As you can see the part ID: xxxxxxxx are the same but I would like to use the search in TL to find ALL posts that have the same "ID: xxxxx" part - to sort out duplicates.

Now I do it manually by sorting the posts and looking at the running time to the left and then check if the ID:xxxxx is the same - very tedious work :(

Is there a regex or other method to find it quickly ?


Top
 Profile  
Reply with quote  
 Post subject: Re: Regex Search For Duplicates
PostPosted: May 21st, 2023, 5:38 am 
Site Admin
User avatar

Joined: March 10th, 2011, 11:14 pm
Posts: 12641
Location: Earth
Hi,

no, I can't think of way to do that more easily in themaLeecher.

That way too specific, it would need a custom script to filter like that.

The only way I can think is to save all posts to .txt files -> use some custom script to filter and delete not needed -> delete all those messages from themaLeecher -> then load back what's left of .txt files to themaLeecher if needed.

_________________
themaPoster | themaCreator | themaManager | themaLeecher | themaRegister


Top
 Profile  
Reply with quote  
 Post subject: Re: Regex Search For Duplicates
PostPosted: May 21st, 2023, 5:43 am 
Site Admin
User avatar

Joined: March 10th, 2011, 11:14 pm
Posts: 12641
Location: Earth
Well, I might actually add a new option to "Match search on duplicates only".

Then it might be possible with regex.

Will take a look more.

_________________
themaPoster | themaCreator | themaManager | themaLeecher | themaRegister


Top
 Profile  
Reply with quote  
 Post subject: Re: Regex Search For Duplicates
PostPosted: May 21st, 2023, 7:01 am 

Joined: November 23rd, 2014, 6:31 am
Posts: 276
Thanks for checking.

The thing here is - I want to keep the duplicates in themaLeecher.
Simply because some creators re-post their content but the videos have same ID - this way I can identify if its a repost.
Also when they re-post - sometimes they delete the old messages or the old messages.
Anyway - I want to keep the posts with same ID

I just need a way to find all posts who have the same ID.

I had my custom RSS changed now so ID NUmber is Second in Subject and runtime is first - then I can sort A-Z but it is still manual checking.

This is how the subject will look:
Code:
 
|32:45| [287164416] Don sudan - 2023.05.16 - bla bla
|33:33| [2828957843] Don sudan - 2023.04.16 - txt xtt
|33:33| [2828957843]Don sudan - 2023.04.16 - dumdidum
|23:33| [14545455] Don sudan - 2023.04.16 - text text

so basically there is now 3 ID´s and I would need a way to filter out to only see the ones with Identical ID in the brackets.

I guess I just hoped there was a regex search string to find duplicate text - imagine 5000 posts or more where I have to look manually :)


Top
 Profile  
Reply with quote  
 Post subject: Re: Regex Search For Duplicates
PostPosted: May 21st, 2023, 2:25 pm 
Site Admin
User avatar

Joined: March 10th, 2011, 11:14 pm
Posts: 12641
Location: Earth
Will try to add a separate option for that.

_________________
themaPoster | themaCreator | themaManager | themaLeecher | themaRegister


Top
 Profile  
Reply with quote  
 Post subject: Re: Regex Search For Duplicates
PostPosted: May 27th, 2023, 11:56 am 
Site Admin
User avatar

Joined: March 10th, 2011, 11:14 pm
Posts: 12641
Location: Earth
4.72:
* Added new search option for messages: "Search text regex matching only duplicates".

For you last example use this regex with that option enable and it will find what you need:
Code:
\[[0-9]+\]

_________________
themaPoster | themaCreator | themaManager | themaLeecher | themaRegister


Top
 Profile  
Reply with quote  
 Post subject: Re: Regex Search For Duplicates
PostPosted: May 29th, 2023, 2:14 pm 

Joined: November 23rd, 2014, 6:31 am
Posts: 276
Thank you! this is really good.

one last regex question regarding this.

I want to find any string of numbers that are duplicates in subject.

Can be [12345] or 12345 or _12345_

is there a solution for that so that its not limited to duplicates withing brackets ?


Top
 Profile  
Reply with quote  
 Post subject: Re: Regex Search For Duplicates
PostPosted: May 29th, 2023, 4:31 pm 
Site Admin
User avatar

Joined: March 10th, 2011, 11:14 pm
Posts: 12641
Location: Earth
Try this:

Code:
.[0-9]+.

_________________
themaPoster | themaCreator | themaManager | themaLeecher | themaRegister


Top
 Profile  
Reply with quote  
 Post subject: Re: Regex Search For Duplicates
PostPosted: May 29th, 2023, 5:03 pm 

Joined: November 23rd, 2014, 6:31 am
Posts: 276
thanks, that did not work for me - it finds subjects that dont have a duplicate string of numbers.


Top
 Profile  
Reply with quote  
 Post subject: Re: Regex Search For Duplicates
PostPosted: May 29th, 2023, 7:49 pm 
Site Admin
User avatar

Joined: March 10th, 2011, 11:14 pm
Posts: 12641
Location: Earth
I would need some messages to test on.

If you can do this: "Help" menu (at the top) -> "Support" -> "Compress" -> send me the file which it opens (via PM in any file host).

Will review your settings.

_________________
themaPoster | themaCreator | themaManager | themaLeecher | themaRegister


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 

Who is online

Users browsing this forum: No registered users and 16 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Theme designed by stylerbb.net © 2008
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
All times are UTC