Generate unique id using strings

Question

I am parsing data from multiple sources and I want to assign a unique (string) id to each entry. Each entry contains a title (string), url(string) and body(string). We can get same title from multiple sources but those will have different urls and I would like to store both the items in that case. I am thinking of creating a hash of title and url and assign that as an id, that ways if I get same title and url from different sources, the id will be same and I will be able to identify that it's a duplicate.

import hashlib 
hashlib.sha256(str("title url").encode('utf-8')).hexdigest()

But I think there can be a case where 2 different title url combinations might generate same hash, not sure how to overcome the clash. Can someone suggest a way of generating unique identifier using strings I don't want to use timestamp because I might get same row from different sources at different times

What about combining title + url + body instead of single title for generating hashes — GooDeeJAY
– GooDeeJAY, Commented Apr 13, 2021 at 8:21
No you won't, there is NO WAY, you get a SHA256 collision, that is a cryptographic hash function, you're safe — azro
– azro, Commented Apr 13, 2021 at 8:21

azro · Accepted Answer · 2021-04-13 08:32:48Z

4

You're safe, you won't have 2 different title url combinations generating same hash with SHA-256

SHA256 is a cryptographic hash function, from the SHA-2 hash family, and is a standard from 2020.

The collision probability (2 inputs gives same output) is 1/(2^128) which is about 2e-39.

See: SHA-256 collisions on crypto.stackexchange

edited Apr 13, 2021 at 8:32

answered Apr 13, 2021 at 8:24

azro

54.2k9 gold badges38 silver badges75 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Generate unique id using strings

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related