Heiko Paulheim, profile picture
Uploaded byHeiko Paulheim
ODP, PPTX1,868 views

Machine Learning & Embeddings for Large Knowledge Graphs

This document discusses machine learning techniques for knowledge graphs. It begins with an overview of typical machine learning tasks involving knowledge graphs, such as type prediction and link prediction. It then discusses challenges in applying traditional machine learning algorithms to knowledge graphs due to their graph-based structure rather than feature vectors. Several techniques are presented to address this, including propositionalization to transform graphs into feature vectors, and knowledge graph embeddings which learn vector representations of entities that preserve structural relationships. Word2vec and its adaptation RDF2vec for knowledge graphs are explained as early embedding techniques, along with subsequent extensions and related methods like TransE.

Embed presentation

Download as ODP, PPTX
7/2/19 Heiko Paulheim 1 Machine Learning & Embeddings for Large Knowledge Graphs Heiko Paulheim
7/2/19 Heiko Paulheim 2 Crossing the Bridge from the Other Side
7/2/19 Heiko Paulheim 3 Crossing the Bridge from the Other Side • There are plenty of established ML and DM toolkits... – Weka – RapidMiner – scikit-learn – R • ...implementing all your favorite algorithms... – Naive Bayes – Random Forests – SVMs – (Deep) Neural Networks – ... • ...but they all work on feature vectors, not graphs!
7/2/19 Heiko Paulheim 4 Typical Tasks • Knowledge Graph Internal – Type prediction – Link prediction – Link validation • Knowledge Graph External – i.e., using the KG as background knowledge in some other task – e.g., content-based recommender systems – e.g., predictive modeling ● who is the next nobel prize winner? Gao et al.: Link Prediction Methods and Their Accuracy for Different Social Networks and Network Metrics. Scientific Programming, 2014 Xu et al.: Explainable Reasoning over Knowledge Graphs for Recommendation. ebay tech blog, 2019
7/2/19 Heiko Paulheim 5 Example: Knowledge Graph Internal • Type prediction – Many instances in KGs are not typed or have very abstract types – e.g., many actors are just typed as persons • Classic approach – Exploit ontology – Shown to be rather sensitive to noise • Example: ontology-based typing of Germany in DBpedia – Airport, Award, Building, City, Country, Ethnic Group, Genre, Language, Military Conflict, Mountain, Mountain Range, Person Function, Place, Populated Place, Race, Route of Transportation, Settlement, Stadium, Wine Region Paulheim & Bizer: Type Inference on Noisy RDF Data. ISWC, 2013 Melo et al.: Type Prediction in Noisy RDF Knowledge Bases using Hierarchical Multilabel Classification with Graph and Latent Features. IJAIT, 2017
7/2/19 Heiko Paulheim 6 Example: Knowledge Graph Internal • Alternative: learn model for type prediction – Train classifier to predict types (binary or hierarchical) – More noise tolerant Paulheim & Bizer: Improving the quality of linked data using statistical distributions. IJSWIS, 2014
7/2/19 Heiko Paulheim 7 Example: Knowledge Graph External • Example machine learning task: predicting book sales ISBN City Sold 3-2347-3427-1 Darmstadt 124 3-43784-324-2 Mannheim 493 3-145-34587-0 Roßdorf 14 ... ISBN City Population ... Genre Publisher ... Sold 3-2347-3427-1 Darm- stadt 144402 ... Crime Bloody Books ... 124 3-43784-324-2 Mann- heim 291458 … Crime Guns Ltd. … 493 3-145-34587-0 Roß- dorf 12019 ... Travel Up&Away ... 14 ... → Crime novels sell better in larger cities Paulheim & Fürnkranz: Unsupervised Generation of Data Mining Features from Linked Open Data. WIMS, 2012
7/2/19 Heiko Paulheim 8 Example: The FeGeLOD Framework IS B N 3 -2 3 4 7 -3 4 2 7 -1 C ity D a r m s ta d t # s o ld 1 2 4 N a m e d E n t it y R e c o g n it io n IS B N 3 -2 3 4 7 -3 4 2 7 - 1 C ity D a r m s ta d t # s o ld 1 2 4 C ity _ U R I h ttp : / / d b p e d ia .o r g / r e s o u r c e/ D a r m s ta d t F e a t u r e G e n e r a t io n IS B N 3 - 2 3 4 7 -3 4 2 7 -1 C ity D a r m s ta d t # s o ld 1 2 4 C ity _ U R I h ttp : / / d b p e d ia .o r g / r e s o u r c e / D a r m s ta d t C ity _ U R I_ d b p e d ia -o w l: p o p u la tio n T o ta l 1 4 1 4 7 1 C ity _ U R I_ ... ... F e a t u r e S e le c t io n IS B N 3 -2 3 4 7 -3 4 2 7 - 1 C ity D a r m s ta d t # s o ld 1 2 4 C ity _ U R I h ttp : / / d b p e d ia .o r g / r e s o u r c e/ D a r m s ta d t C ity _ U R I_ d b p e d ia -o w l:p o p u la tio n T o ta l 1 4 1 4 7 1 Paulheim & Fürnkranz: Unsupervised Generation of Data Mining Features from Linked Open Data. WIMS, 2012
7/2/19 Heiko Paulheim 9 The FeGeLOD Framework • Entity Recognition – Simple approach: guess DBpedia URIs – Hit rate >95% for cities and countries (by English name) • Feature Generation – augmenting the dataset with additional attributes from KG • Feature Selection – Filter noise: >95% unknown, identical, or different nominals Paulheim & Fürnkranz: Unsupervised Generation of Data Mining Features from Linked Open Data. WIMS, 2012
7/2/19 Heiko Paulheim 10 Propositionalization • Bridge Problem: Knowledge Graphs vs. ML algorithms expecting Feature Vectors → wanted: a transformation from nodes to sets of features ? Ristoski & Paulheim: A Comparison of Propositionalization Strategies for Creating Features from Linked Open Data. LD4KD, 2014
7/2/19 Heiko Paulheim 11 Propositionalization • Bridge Problem: Knowledge Graphs vs. ML algorithms expecting Feature Vectors → wanted: a transformation from nodes to sets of features • Basic strategies: – literal values (e.g., population) are used directly – instance types become binary features – relations are counted (absolute, relative, TF-IDF) – combinations of relations and object types are counted (absolute, relative, TF-IDF) – ... Ristoski & Paulheim: A Comparison of Propositionalization Strategies for Creating Features from Linked Open Data. LD4KD, 2014
7/2/19 Heiko Paulheim 12 Propositionalization ctd. • Observations – Even simple features (e.g., add all numbers and types) can help on many problems – More sophisticated features often bring additional improvements ● Combinations of relations and individuals – e.g., movies directed by Steven Spielberg ● Combinations of relations and types – e.g., movies directed by Oscar-winning directors ● … – But ● The search space is enormous! ● Generate first, filter later does not scale well Ristoski & Paulheim: A Comparison of Propositionalization Strategies for Creating Features from Linked Open Data. LD4KD, 2014
7/2/19 Heiko Paulheim 13 From Naive Propositionalization to Knowledge Graph Embeddings • Reconsidering the previous examples: – We want to predict some attribute of a KG entity ● e.g., types ● e.g., sales figures of books – ...given the entity’s vector representation • How do we get a “good” vector representation for an entity? – ...and: what is “good” in the first place?
7/2/19 Heiko Paulheim 14 From Naive Propositionalization to Knowledge Graph Embeddings • How do we get a “good” vector representation for an entity? – ...and: what is “good” in the first place? • “good” for machine learning means separable – similar entities are close together – different entities are further away https://appliedmachinelearning.blog/2017/03/09/understanding-support-vector-machines-a-primer/
7/2/19 Heiko Paulheim 15 A Brief Excursion to word2vec • A vector space model for words • Introduced in 2013 • Each word becomes a vector – similar words are close – relations are preserved – vector arithmetics are possible https://www.adityathakker.com/introduction-to-word2vec-how-it-works/
7/2/19 Heiko Paulheim 16 A Brief Excursion to word2vec • Assumption: – Similar words appear in similar contexts {Bush,Obama,Trump} was elected president of the United States United States president {Bush,Obama,Trump} announced… … • Idea – Train a network that can predict a word from its context (CBOW) or the context from a word (Skip Gram) Mikolov et al.: Efficient Estimation of Word Representations in Vector Space. 2013
7/2/19 Heiko Paulheim 17 A Brief Excursion to word2vec • Skip Gram: train a neural network with one hidden layer • Use output values at hidden layer as vector representation • Observation: – Bush, Obama, Trump will activate similar context words – i.e., their output weights at the projection layer have to be similar Mikolov et al.: Efficient Estimation of Word Representations in Vector Space. 2013
7/2/19 Heiko Paulheim 18 From word2vec to RDF2vec • Word2vec operates on sentences, i.e., sequences of words • Idea of RDF2vec – First extract “sentences” from a graph – Then train embedding using RDF2vec • “Sentences” are extracted by performing random graph walks: Year Zero Nine Inch Nails Trent Reznor • Experiments – RDF2vec can be trained on large KGs (DBpedia, Wikidata) – 300-500 dimensional vectors outperform other propositionalization strategies artist member Ristoski & Paulheim: RDF2vec: RDF Graph Embeddings for Data Mining. ISWC, 2016
7/2/19 Heiko Paulheim 19 From word2vec to RDF2vec • RDF2vec example – similar instances form clusters – direction of relations is stable Ristoski & Paulheim: RDF2vec: RDF Graph Embeddings for Data Mining. ISWC, 2016
7/2/19 Heiko Paulheim 20 From word2vec to RDF2vec • RecSys example: using proximity in latent RDF2vec feature space Ristoski et al.: RDF2Vec: RDF Graph Embeddings and their Applications. SWJ 10(4), 2019
7/2/19 Heiko Paulheim 21 Extensions of RDF2vec • Maybe random walks are not such a good idea – They may give too much weight on less-known entities and facts ● Strategies: – Prefer edges with more frequent predicates – Prefer nodes with higher indegree – Prefer nodes with higher PageRank – … – They may cover less-known entities and facts too little ● Strategies: – The opposite of all of the above strategies • Bottom line of experimental evaluation: – Not one strategy fits all Cochez et al.: Biased Graph Walks for RDF Graph Embeddings. WIMS, 2017
7/2/19 Heiko Paulheim 22 Other Word Embedding Methods • GloVe (Global Word Embedding Vectors) • Computes embeddings out of co-occurence statistics – Using matrix factorization • Has been applied to random RDF walks as well • Experimental evaluation: – In some cases, RDFGloVe outperforms RDF2vec https://www.kdnuggets.com/2018/04/implementing-deep-learning-methods-feature-engineering-text-data- glove.html Cochez et al.: Global RDF Vector Space Embeddings, ISWC, 2017
7/2/19 Heiko Paulheim 23 Other Word Embedding Methods • There is a lot of promising stuff not yet tried – e.g., biasing walks based on human factors – e.g., more recent word embedding methods such as ELMo and BERT https://www.nbcnews.com/feature/nbc-out/bert-ernie-are-gay-couple-sesame-street-writer-claims-n910701
7/2/19 Heiko Paulheim 24 TransE and its Descendants • In RDF2vec, relation preservation is a by-product • TransE: direct modeling – Formulates RDF embedding as an optimization problem – Find mapping of entities and relations to Rn so that ● across all triples <s,p,o> Σ ||s+p-o|| is minimized ● try to obtain a smaller error for existing triples than for non-existing ones Bordes et al: Translating Embeddings for Modeling Multi-relational Data. NIPS 2013. Fan et al.: Learning Embedding Representations for Knowledge Inference on Imperfect and Incomplete Repositories. WI 2016
7/2/19 Heiko Paulheim 25 Limitations of TransE • Symmetric properties – we have to minimize ||Barack + spouse – Michelle|| and ||Michelle + spouse – Barack|| simultaneously – ideally, Barack + spouse = Michelle and Michelle + spouse = Barack ● Michelle and Barack become infinitely close ● spouse becomes 0 vector Michelle Barack
7/2/19 Heiko Paulheim 26 Limitations of TransE • Transitive Properties – we have to minimize ||Miami + partOf – Florida|| and ||Florida + partOf – USA||, but also ||Miami + partOf – USA|| – ideally, Miami + partOf = Florida, Florida + partOf = USA, Miami + partOf = USA ● Again: all three become infinitely close ● partOf becomes 0 vector Florida Miami USA
7/2/19 Heiko Paulheim 27 Limitations of TransE • One to many properties – we have to minimize ||New York + partOf – USA||, ||Florida + partOf – USA||, ||Ohio + partOf – USA||, … – ideally, NewYork + partOf = USA, Florida + partOf = USA, Ohio + partOf = USA ● all the subjects become infinitely close Florida USA New York Ohio
7/2/19 Heiko Paulheim 28 Limitations of TransE • Reflexive properties – we have to minimize ||Tom + knows - Tom|| – ideally, Tom + knows = Tom ● Knows becomes 0 vector Tom
7/2/19 Heiko Paulheim 29 TransE RDF2Vec HolE DistMult RESCAL NTN TransR TransH TransD KG2E ComplEx Limitations of TransE • Numerous variants of TransE have been proposed to overcome limitations (e.g., TransH, TransR, TransD, …) • Plus: embedding approaches based on tensor factorization etc.
7/2/19 Heiko Paulheim 30 Are we Driving on the Wrong Side of the Road?
7/2/19 Heiko Paulheim 31 Are we Driving on the Wrong Side of the Road? • Original ideas: – Assign meaning to data – Allow for machine inference – Explain inference results to the user Berners-Lee et al: The Semantic Web. Scientific American, May 2001
7/2/19 Heiko Paulheim 32 Running Example: Recommender Systems • Content based recommender systems backed by Semantic Web data – (today: knowledge graphs) • Advantages – use rich background information about recommended items (for free) – justifications can be generated (e.g., you like movies by that director) https://lazyprogrammer.me/tutorial-on-collaborative-filtering-and-matrix-factorization-in-python/
7/2/19 Heiko Paulheim 33 The 2009 Semantic Web Layer Cake
7/2/19 Heiko Paulheim 34 The 2019 Semantic Web Layer Cake Embeddings
7/2/19 Heiko Paulheim 35 Towards Semantic Vector Space Embeddings cartoon superhero Ristoski et al.: RDF2Vec: RDF Graph Embeddings and their Applications. SWJ 10(4), 2019
7/2/19 Heiko Paulheim 36 The Holy Grail • Combine semantics and embeddings – e.g., directly create meaningful dimensions – e.g., learn interpretation of dimensions a posteriori – ...
7/2/19 Heiko Paulheim 37 A New Design Space quantitative performance semantic interpretability
7/2/19 Heiko Paulheim 38 Software to Check Out • http://openke.thunlp.org/ – Implements many embedding approaches – Pre-trained vectors available, e.g., for Wikidata
7/2/19 Heiko Paulheim 39 Software to Check Out • Loading RDF in Python: https://github.com/RDFLib/rdflib
7/2/19 Heiko Paulheim 40 RapidMiner Linked Open Data Extension caution: works only until RM6! :-(
7/2/19 Heiko Paulheim 41 References (1) • Berners-Lee, T., Hendler, J., & Lassila, O. (2001). The semantic web. Scientific american, 284(5), 28-37. • Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., & Yakhnenko, O. (2013). Translating embeddings for modeling multi-relational data. In NIPS (pp. 2787-2795). • Cochez, M., Ristoski, P., Ponzetto, S. P., & Paulheim, H. (2017). Biased graph walks for RDF graph embeddings. In WIMS (p. 21). ACM. • Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. • Melo, A., Völker, J., & Paulheim, H. (2017). Type prediction in noisy RDF knowledge bases using hierarchical multilabel classification with graph and latent features. IJAIT, 26(02). • Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. • Paulheim, H., & Fümkranz, J. (2012). Unsupervised generation of data mining features from linked open data. In WIMS (p. 31). ACM. • Paulheim, H., & Bizer, C. (2013). Type inference on noisy RDF data. In International semantic web conference (pp. 510-525). Springer, Berlin, Heidelberg.
7/2/19 Heiko Paulheim 42 References (2) • Paulheim, H., & Bizer, C. (2014). Improving the quality of linked data using statistical distributions. IJSWIS, 10(2), 63-86. • Paulheim, H. (2017). Knowledge graph refinement: A survey of approaches and evaluation methods. Semantic web, 8(3), 489-508. • Paulheim, H. (2018). Make Embeddings Semantic Again! ISWC (Blue Sky Track) • Peters, M. E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018). Deep contextualized word representations. arXiv preprint arXiv:1802.05365. • Ristoski, P., & Paulheim, H. (2014). A comparison of propositionalization strategies for creating features from linked open data. Linked Data for Knowledge Discovery, 6. • Ristoski, P., Bizer, C., & Paulheim, H. (2015). Mining the web of linked data with rapidminer. Web Semantics: Science, Services and Agents on the World Wide Web, 35, 142-151. • Ristoski, P., & Paulheim, H. (2016). Semantic Web in data mining and knowledge discovery: A comprehensive survey. Web semantics, 36, 1-22. • Ristoski, P., & Paulheim, H. (2016). RDF2vec: RDF graph embeddings for data mining. In International Semantic Web Conference (pp. 498-514). Springer, Cham. • Ristoski, P., Rosati, J., Di Noia, T., De Leone, R., & Paulheim, H. (2019). RDF2Vec: RDF graph embeddings and their applications. Semantic Web, 10(4), 1-32.
7/2/19 Heiko Paulheim 43 Machine Learning & Embeddings for Large Knowledge Graphs Heiko Paulheim

More Related Content

PPTX
Practical contextual bandits for business
byYan Xu
 
PDF
Deep Learning for Recommender Systems
byJustin Basilico
 
PDF
Python For Data Analysis | Python Pandas Tutorial | Learn Python | Python Tra...
byEdureka!
 
PDF
Tutorial on Deep Learning in Recommender System, Lars summer school 2019
byAnoop Deoras
 
PDF
Learning to Rank for Recommender Systems - ACM RecSys 2013 tutorial
byAlexandros Karatzoglou
 
PPTX
Neural Text Embeddings for Information Retrieval (WSDM 2017)
byBhaskar Mitra
 
PDF
Knowledge graphs, meet Deep Learning
byConnected Data World
 
PDF
ML Infra for Netflix Recommendations - AI NEXTCon talk
byFaisal Siddiqi
 
Practical contextual bandits for business
byYan Xu
 
Deep Learning for Recommender Systems
byJustin Basilico
 
Python For Data Analysis | Python Pandas Tutorial | Learn Python | Python Tra...
byEdureka!
 
Tutorial on Deep Learning in Recommender System, Lars summer school 2019
byAnoop Deoras
 
Learning to Rank for Recommender Systems - ACM RecSys 2013 tutorial
byAlexandros Karatzoglou
 
Neural Text Embeddings for Information Retrieval (WSDM 2017)
byBhaskar Mitra
 
Knowledge graphs, meet Deep Learning
byConnected Data World
 
ML Infra for Netflix Recommendations - AI NEXTCon talk
byFaisal Siddiqi
 

What's hot

PDF
Ehtsham Elahi, Senior Research Engineer, Personalization Science and Engineer...
byMLconf
 
PPTX
Netflix talk at ML Platform meetup Sep 2019
byFaisal Siddiqi
 
PPTX
GRU4Rec v2 - Recurrent Neural Networks with Top-k Gains for Session-based Rec...
byBalázs Hidasi
 
PDF
Matrix Factorization
byYusuke Yamamoto
 
PDF
Recent Trends in Personalization: A Netflix Perspective
byJustin Basilico
 
PDF
Recommendation System Explained
byCrossing Minds
 
PDF
Real-Time Recommendations with Hopsworks and OpenSearch - MLOps World 2022
byJim Dowling
 
PPT
Recommendation system
byVikrant Arya
 
PDF
Graph Neural Networks for Recommendations
byWQ Fan
 
PDF
Sequential Decision Making in Recommendations
byJaya Kawale
 
PPTX
Recommendation system
byAkshat Thakar
 
PPTX
A Simple Introduction to Neural Information Retrieval
byBhaskar Mitra
 
PDF
Meta-Prod2Vec: Simple Product Embeddings with Side-Information
byrecsysfr
 
PPTX
ATO 2022 - Machine Learning + Graph Databases for Better Recommendations (3)....
byArangoDB Database
 
PDF
Time, Context and Causality in Recommender Systems
byYves Raimond
 
PDF
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...
byXavier Amatriain
 
PDF
Past, present, and future of Recommender Systems: an industry perspective
byXavier Amatriain
 
PDF
Kdd 2014 Tutorial - the recommender problem revisited
byXavier Amatriain
 
PDF
Intro to LLMs
byLoic Merckel
 
ODP
Topic Modeling
byKarol Grzegorczyk
 
Ehtsham Elahi, Senior Research Engineer, Personalization Science and Engineer...
byMLconf
 
Netflix talk at ML Platform meetup Sep 2019
byFaisal Siddiqi
 
GRU4Rec v2 - Recurrent Neural Networks with Top-k Gains for Session-based Rec...
byBalázs Hidasi
 
Matrix Factorization
byYusuke Yamamoto
 
Recent Trends in Personalization: A Netflix Perspective
byJustin Basilico
 
Recommendation System Explained
byCrossing Minds
 
Real-Time Recommendations with Hopsworks and OpenSearch - MLOps World 2022
byJim Dowling
 
Recommendation system
byVikrant Arya
 
Graph Neural Networks for Recommendations
byWQ Fan
 
Sequential Decision Making in Recommendations
byJaya Kawale
 
Recommendation system
byAkshat Thakar
 
A Simple Introduction to Neural Information Retrieval
byBhaskar Mitra
 
Meta-Prod2Vec: Simple Product Embeddings with Side-Information
byrecsysfr
 
ATO 2022 - Machine Learning + Graph Databases for Better Recommendations (3)....
byArangoDB Database
 
Time, Context and Causality in Recommender Systems
byYves Raimond
 
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...
byXavier Amatriain
 
Past, present, and future of Recommender Systems: an industry perspective
byXavier Amatriain
 
Kdd 2014 Tutorial - the recommender problem revisited
byXavier Amatriain
 
Intro to LLMs
byLoic Merckel
 
Topic Modeling
byKarol Grzegorczyk
 

Similar to Machine Learning & Embeddings for Large Knowledge Graphs

PDF
New Adventures in RDF2vec
byHeiko Paulheim
 
PDF
New Adventures in RDF2vec
byHeiko Paulheim
 
PDF
What_do_Knowledge_Graph_Embeddings_Learn.pdf
byHeiko Paulheim
 
PPTX
RDF2Vec: RDF Graph Embeddings for Data Mining
byPetar Ristoski
 
PDF
Using Knowledge Graphs in Data Science - From Symbolic to Latent Representati...
byHeiko Paulheim
 
PDF
An Empirical Comparison of Knowledge Graph Embeddings for Item Recommendation
byEnrico Palumbo
 
PPTX
PyData Los Angeles 2020 (Abhilash Majumder)
byAbhilash Majumder
 
PDF
Creating Community at WeWork through Graph Embeddings with node2vec - Karry Lu
byRising Media Ltd.
 
PPTX
Neural Models for Information Retrieval
byBhaskar Mitra
 
PDF
A Study of the Similarities of Entity Embeddings Learned from Different Aspec...
byGUANGYUAN PIAO
 
PDF
kdd_talk.pdf
byssuser6d9950
 
PDF
kdd_talk.pdf
byssuser6d9950
 
PPTX
Deepwalk vs Node2vec
bySiddhantVerma49
 
PDF
Texts Classification with the usage of Neural Network based on the Word2vec’s...
byijsc
 
PDF
Texts Classification with the usage of Neural Network based on the Word2vec’s...
byijsc
 
PDF
TEXTS CLASSIFICATION WITH THE USAGE OF NEURAL NETWORK BASED ON THE WORD2VEC’S...
byijsc
 
PPTX
node2vec: Scalable Feature Learning for Networks.pptx
byssuser2624f71
 
PDF
Top Cited Articles in Data Mining - International Journal of Data Mining & Kn...
byIJDKP
 
ODP
Exploiting Linked Open Data as Background Knowledge in Data Mining
byHeiko Paulheim
 
PDF
Deep Learning & NLP: Graphs to the Rescue!
byRoelof Pieters
 
New Adventures in RDF2vec
byHeiko Paulheim
 
New Adventures in RDF2vec
byHeiko Paulheim
 
What_do_Knowledge_Graph_Embeddings_Learn.pdf
byHeiko Paulheim
 
RDF2Vec: RDF Graph Embeddings for Data Mining
byPetar Ristoski
 
Using Knowledge Graphs in Data Science - From Symbolic to Latent Representati...
byHeiko Paulheim
 
An Empirical Comparison of Knowledge Graph Embeddings for Item Recommendation
byEnrico Palumbo
 
PyData Los Angeles 2020 (Abhilash Majumder)
byAbhilash Majumder
 
Creating Community at WeWork through Graph Embeddings with node2vec - Karry Lu
byRising Media Ltd.
 
Neural Models for Information Retrieval
byBhaskar Mitra
 
A Study of the Similarities of Entity Embeddings Learned from Different Aspec...
byGUANGYUAN PIAO
 
kdd_talk.pdf
byssuser6d9950
 
kdd_talk.pdf
byssuser6d9950
 
Deepwalk vs Node2vec
bySiddhantVerma49
 
Texts Classification with the usage of Neural Network based on the Word2vec’s...
byijsc
 
Texts Classification with the usage of Neural Network based on the Word2vec’s...
byijsc
 
TEXTS CLASSIFICATION WITH THE USAGE OF NEURAL NETWORK BASED ON THE WORD2VEC’S...
byijsc
 
node2vec: Scalable Feature Learning for Networks.pptx
byssuser2624f71
 
Top Cited Articles in Data Mining - International Journal of Data Mining & Kn...
byIJDKP
 
Exploiting Linked Open Data as Background Knowledge in Data Mining
byHeiko Paulheim
 
Deep Learning & NLP: Graphs to the Rescue!
byRoelof Pieters
 

More from Heiko Paulheim

PDF
Knowledge Graph Generation from Wikipedia in the Age of ChatGPT: Knowledge ...
byHeiko Paulheim
 
PDF
Knowledge Matters! The Role of Knowledge Graphs in Modern AI Systems
byHeiko Paulheim
 
PDF
From Wikis to Knowledge Graphs
byHeiko Paulheim
 
PPT
Beyond DBpedia and YAGO – The New Kids on the Knowledge Graph Block
byHeiko Paulheim
 
PPT
Big Data, Smart Algorithms, and Market Power - A Computer Scientist’s Perspec...
byHeiko Paulheim
 
ODP
From Wikipedia to Thousands of Wikis – The DBkWik Knowledge Graph
byHeiko Paulheim
 
ODP
Big Data, Smart Algorithms, and Market Power - A Computer Scientist's Perspec...
byHeiko Paulheim
 
ODP
Make Embeddings Semantic Again!
byHeiko Paulheim
 
ODP
How much is a Triple?
byHeiko Paulheim
 
ODP
Machine Learning with and for Semantic Web Knowledge Graphs
byHeiko Paulheim
 
ODP
Weakly Supervised Learning for Fake News Detection on Twitter
byHeiko Paulheim
 
PDF
Towards Knowledge Graph Profiling
byHeiko Paulheim
 
ODP
Knowledge Graphs on the Web
byHeiko Paulheim
 
ODP
Data-driven Joint Debugging of the DBpedia Mappings and Ontology
byHeiko Paulheim
 
ODP
Fast Approximate A-box Consistency Checking using Machine Learning
byHeiko Paulheim
 
PPT
Serving DBpedia with DOLCE - More Than Just Adding a Cherry on Top
byHeiko Paulheim
 
ODP
Combining Ontology Matchers via Anomaly Detection
byHeiko Paulheim
 
PPT
Gathering Alternative Surface Forms for DBpedia Entities
byHeiko Paulheim
 
ODP
What the Adoption of schema.org Tells about Linked Open Data
byHeiko Paulheim
 
ODP
Linked Open Data enhanced Knowledge Discovery
byHeiko Paulheim
 
Knowledge Graph Generation from Wikipedia in the Age of ChatGPT: Knowledge ...
byHeiko Paulheim
 
Knowledge Matters! The Role of Knowledge Graphs in Modern AI Systems
byHeiko Paulheim
 
From Wikis to Knowledge Graphs
byHeiko Paulheim
 
Beyond DBpedia and YAGO – The New Kids on the Knowledge Graph Block
byHeiko Paulheim
 
Big Data, Smart Algorithms, and Market Power - A Computer Scientist’s Perspec...
byHeiko Paulheim
 
From Wikipedia to Thousands of Wikis – The DBkWik Knowledge Graph
byHeiko Paulheim
 
Big Data, Smart Algorithms, and Market Power - A Computer Scientist's Perspec...
byHeiko Paulheim
 
Make Embeddings Semantic Again!
byHeiko Paulheim
 
How much is a Triple?
byHeiko Paulheim
 
Machine Learning with and for Semantic Web Knowledge Graphs
byHeiko Paulheim
 
Weakly Supervised Learning for Fake News Detection on Twitter
byHeiko Paulheim
 
Towards Knowledge Graph Profiling
byHeiko Paulheim
 
Knowledge Graphs on the Web
byHeiko Paulheim
 
Data-driven Joint Debugging of the DBpedia Mappings and Ontology
byHeiko Paulheim
 
Fast Approximate A-box Consistency Checking using Machine Learning
byHeiko Paulheim
 
Serving DBpedia with DOLCE - More Than Just Adding a Cherry on Top
byHeiko Paulheim
 
Combining Ontology Matchers via Anomaly Detection
byHeiko Paulheim
 
Gathering Alternative Surface Forms for DBpedia Entities
byHeiko Paulheim
 
What the Adoption of schema.org Tells about Linked Open Data
byHeiko Paulheim
 
Linked Open Data enhanced Knowledge Discovery
byHeiko Paulheim
 

Recently uploaded

DOCX
THESIS ABSTRACT ABOUT CHALLENGES ENCOUNTERED BY POLICE OFFICERS.docx
byssuser6a111b
 
PDF
UG ECE SYLLABUS 2022-26 batchUG ECE SYLLABUS 2022-26 batch
byECENVSatyanarayanaMu
 
PPTX
Python for Data Analysis & Decision Making.pptx
bydasthagirihachion
 
PPTX
STATISTICAL TECHNIQUES USING SPSS SOFTWARE
bysaberunnisaa
 
PPTX
Holiday Class Agenda Education Presentation in Pink Red and Green Simple Orna...
byAngelDebatian
 
PPTX
Advanced Python Course With Certification
byinderjeetsingh0589
 
PPTX
SAC 28 Finals Data Insights of Startup Investing 2025 1119.pptx
byElaine Werffeli
 
PPTX
English Presentation automation -Why technology is replacing jobs?
byrifatnot404
 
PDF
Document 2.pdf bizimkiler yapmış yine dvfweefsdc
byAlperturkoglu
 
PPTX
Recurrent-Neural-Networks-RNNs.pptx (1).pptx
byharbingernoobsaibot
 
PPT
PETROLEUM TRAPS.pptPETROLEUM TRAPS.pptmb
byalialialsatre
 
PDF
[Redis Released]- FalkorDB - Redis + Graph Agentic Memory’s Secret Sauce
byGuy Korland
 
PDF
Advanced Data Automation, Orchestration & Cloud Analytics | Wisecor Transform...
bywisecortransformatio
 
PPTX
Basics of Research Methodology Part1.pptx
byDepartment of Agricultural Marketing
 
PPTX
HARNESSING R: STATISTICAL TECHNIQUES FOR DATA-DRIVEN ENTREPRENEURSHIP
bysaberunnisaa
 
PPT
welcometocomputernetworks-100907004628-phpapp02.ppt
byssuser6c0026
 
PDF
IDENTIFICATION OF ARMA MODELSSSSSSSSSSSS
byilgazduman1
 
PDF
UG Syllabus 2023-24 UG Syllabus 2023-24UG Syllabus 2023-24
byECENVSatyanarayanaMu
 
PPTX
Correlation-Regression analysis -16.11.25.pptx
byFahmida Swati
 
PDF
Mari menyusun kegiatan yang memberikan makna
byMcDonaldPorajow1
 
THESIS ABSTRACT ABOUT CHALLENGES ENCOUNTERED BY POLICE OFFICERS.docx
byssuser6a111b
 
UG ECE SYLLABUS 2022-26 batchUG ECE SYLLABUS 2022-26 batch
byECENVSatyanarayanaMu
 
Python for Data Analysis & Decision Making.pptx
bydasthagirihachion
 
STATISTICAL TECHNIQUES USING SPSS SOFTWARE
bysaberunnisaa
 
Holiday Class Agenda Education Presentation in Pink Red and Green Simple Orna...
byAngelDebatian
 
Advanced Python Course With Certification
byinderjeetsingh0589
 
SAC 28 Finals Data Insights of Startup Investing 2025 1119.pptx
byElaine Werffeli
 
English Presentation automation -Why technology is replacing jobs?
byrifatnot404
 
Document 2.pdf bizimkiler yapmış yine dvfweefsdc
byAlperturkoglu
 
Recurrent-Neural-Networks-RNNs.pptx (1).pptx
byharbingernoobsaibot
 
PETROLEUM TRAPS.pptPETROLEUM TRAPS.pptmb
byalialialsatre
 
[Redis Released]- FalkorDB - Redis + Graph Agentic Memory’s Secret Sauce
byGuy Korland
 
Advanced Data Automation, Orchestration & Cloud Analytics | Wisecor Transform...
bywisecortransformatio
 
Basics of Research Methodology Part1.pptx
byDepartment of Agricultural Marketing
 
HARNESSING R: STATISTICAL TECHNIQUES FOR DATA-DRIVEN ENTREPRENEURSHIP
bysaberunnisaa
 
welcometocomputernetworks-100907004628-phpapp02.ppt
byssuser6c0026
 
IDENTIFICATION OF ARMA MODELSSSSSSSSSSSS
byilgazduman1
 
UG Syllabus 2023-24 UG Syllabus 2023-24UG Syllabus 2023-24
byECENVSatyanarayanaMu
 
Correlation-Regression analysis -16.11.25.pptx
byFahmida Swati
 
Mari menyusun kegiatan yang memberikan makna
byMcDonaldPorajow1
 

Machine Learning & Embeddings for Large Knowledge Graphs

  • 1.
    7/2/19 Heiko Paulheim1 Machine Learning & Embeddings for Large Knowledge Graphs Heiko Paulheim
  • 2.
    7/2/19 Heiko Paulheim2 Crossing the Bridge from the Other Side
  • 3.
    7/2/19 Heiko Paulheim3 Crossing the Bridge from the Other Side • There are plenty of established ML and DM toolkits... – Weka – RapidMiner – scikit-learn – R • ...implementing all your favorite algorithms... – Naive Bayes – Random Forests – SVMs – (Deep) Neural Networks – ... • ...but they all work on feature vectors, not graphs!
  • 4.
    7/2/19 Heiko Paulheim4 Typical Tasks • Knowledge Graph Internal – Type prediction – Link prediction – Link validation • Knowledge Graph External – i.e., using the KG as background knowledge in some other task – e.g., content-based recommender systems – e.g., predictive modeling ● who is the next nobel prize winner? Gao et al.: Link Prediction Methods and Their Accuracy for Different Social Networks and Network Metrics. Scientific Programming, 2014 Xu et al.: Explainable Reasoning over Knowledge Graphs for Recommendation. ebay tech blog, 2019
  • 5.
    7/2/19 Heiko Paulheim5 Example: Knowledge Graph Internal • Type prediction – Many instances in KGs are not typed or have very abstract types – e.g., many actors are just typed as persons • Classic approach – Exploit ontology – Shown to be rather sensitive to noise • Example: ontology-based typing of Germany in DBpedia – Airport, Award, Building, City, Country, Ethnic Group, Genre, Language, Military Conflict, Mountain, Mountain Range, Person Function, Place, Populated Place, Race, Route of Transportation, Settlement, Stadium, Wine Region Paulheim & Bizer: Type Inference on Noisy RDF Data. ISWC, 2013 Melo et al.: Type Prediction in Noisy RDF Knowledge Bases using Hierarchical Multilabel Classification with Graph and Latent Features. IJAIT, 2017
  • 6.
    7/2/19 Heiko Paulheim6 Example: Knowledge Graph Internal • Alternative: learn model for type prediction – Train classifier to predict types (binary or hierarchical) – More noise tolerant Paulheim & Bizer: Improving the quality of linked data using statistical distributions. IJSWIS, 2014
  • 7.
    7/2/19 Heiko Paulheim7 Example: Knowledge Graph External • Example machine learning task: predicting book sales ISBN City Sold 3-2347-3427-1 Darmstadt 124 3-43784-324-2 Mannheim 493 3-145-34587-0 Roßdorf 14 ... ISBN City Population ... Genre Publisher ... Sold 3-2347-3427-1 Darm- stadt 144402 ... Crime Bloody Books ... 124 3-43784-324-2 Mann- heim 291458 … Crime Guns Ltd. … 493 3-145-34587-0 Roß- dorf 12019 ... Travel Up&Away ... 14 ... → Crime novels sell better in larger cities Paulheim & Fürnkranz: Unsupervised Generation of Data Mining Features from Linked Open Data. WIMS, 2012
  • 8.
    7/2/19 Heiko Paulheim8 Example: The FeGeLOD Framework IS B N 3 -2 3 4 7 -3 4 2 7 -1 C ity D a r m s ta d t # s o ld 1 2 4 N a m e d E n t it y R e c o g n it io n IS B N 3 -2 3 4 7 -3 4 2 7 - 1 C ity D a r m s ta d t # s o ld 1 2 4 C ity _ U R I h ttp : / / d b p e d ia .o r g / r e s o u r c e/ D a r m s ta d t F e a t u r e G e n e r a t io n IS B N 3 - 2 3 4 7 -3 4 2 7 -1 C ity D a r m s ta d t # s o ld 1 2 4 C ity _ U R I h ttp : / / d b p e d ia .o r g / r e s o u r c e / D a r m s ta d t C ity _ U R I_ d b p e d ia -o w l: p o p u la tio n T o ta l 1 4 1 4 7 1 C ity _ U R I_ ... ... F e a t u r e S e le c t io n IS B N 3 -2 3 4 7 -3 4 2 7 - 1 C ity D a r m s ta d t # s o ld 1 2 4 C ity _ U R I h ttp : / / d b p e d ia .o r g / r e s o u r c e/ D a r m s ta d t C ity _ U R I_ d b p e d ia -o w l:p o p u la tio n T o ta l 1 4 1 4 7 1 Paulheim & Fürnkranz: Unsupervised Generation of Data Mining Features from Linked Open Data. WIMS, 2012
  • 9.
    7/2/19 Heiko Paulheim9 The FeGeLOD Framework • Entity Recognition – Simple approach: guess DBpedia URIs – Hit rate >95% for cities and countries (by English name) • Feature Generation – augmenting the dataset with additional attributes from KG • Feature Selection – Filter noise: >95% unknown, identical, or different nominals Paulheim & Fürnkranz: Unsupervised Generation of Data Mining Features from Linked Open Data. WIMS, 2012
  • 10.
    7/2/19 Heiko Paulheim10 Propositionalization • Bridge Problem: Knowledge Graphs vs. ML algorithms expecting Feature Vectors → wanted: a transformation from nodes to sets of features ? Ristoski & Paulheim: A Comparison of Propositionalization Strategies for Creating Features from Linked Open Data. LD4KD, 2014
  • 11.
    7/2/19 Heiko Paulheim11 Propositionalization • Bridge Problem: Knowledge Graphs vs. ML algorithms expecting Feature Vectors → wanted: a transformation from nodes to sets of features • Basic strategies: – literal values (e.g., population) are used directly – instance types become binary features – relations are counted (absolute, relative, TF-IDF) – combinations of relations and object types are counted (absolute, relative, TF-IDF) – ... Ristoski & Paulheim: A Comparison of Propositionalization Strategies for Creating Features from Linked Open Data. LD4KD, 2014
  • 12.
    7/2/19 Heiko Paulheim12 Propositionalization ctd. • Observations – Even simple features (e.g., add all numbers and types) can help on many problems – More sophisticated features often bring additional improvements ● Combinations of relations and individuals – e.g., movies directed by Steven Spielberg ● Combinations of relations and types – e.g., movies directed by Oscar-winning directors ● … – But ● The search space is enormous! ● Generate first, filter later does not scale well Ristoski & Paulheim: A Comparison of Propositionalization Strategies for Creating Features from Linked Open Data. LD4KD, 2014
  • 13.
    7/2/19 Heiko Paulheim13 From Naive Propositionalization to Knowledge Graph Embeddings • Reconsidering the previous examples: – We want to predict some attribute of a KG entity ● e.g., types ● e.g., sales figures of books – ...given the entity’s vector representation • How do we get a “good” vector representation for an entity? – ...and: what is “good” in the first place?
  • 14.
    7/2/19 Heiko Paulheim14 From Naive Propositionalization to Knowledge Graph Embeddings • How do we get a “good” vector representation for an entity? – ...and: what is “good” in the first place? • “good” for machine learning means separable – similar entities are close together – different entities are further away https://appliedmachinelearning.blog/2017/03/09/understanding-support-vector-machines-a-primer/
  • 15.
    7/2/19 Heiko Paulheim15 A Brief Excursion to word2vec • A vector space model for words • Introduced in 2013 • Each word becomes a vector – similar words are close – relations are preserved – vector arithmetics are possible https://www.adityathakker.com/introduction-to-word2vec-how-it-works/
  • 16.
    7/2/19 Heiko Paulheim16 A Brief Excursion to word2vec • Assumption: – Similar words appear in similar contexts {Bush,Obama,Trump} was elected president of the United States United States president {Bush,Obama,Trump} announced… … • Idea – Train a network that can predict a word from its context (CBOW) or the context from a word (Skip Gram) Mikolov et al.: Efficient Estimation of Word Representations in Vector Space. 2013
  • 17.
    7/2/19 Heiko Paulheim17 A Brief Excursion to word2vec • Skip Gram: train a neural network with one hidden layer • Use output values at hidden layer as vector representation • Observation: – Bush, Obama, Trump will activate similar context words – i.e., their output weights at the projection layer have to be similar Mikolov et al.: Efficient Estimation of Word Representations in Vector Space. 2013
  • 18.
    7/2/19 Heiko Paulheim18 From word2vec to RDF2vec • Word2vec operates on sentences, i.e., sequences of words • Idea of RDF2vec – First extract “sentences” from a graph – Then train embedding using RDF2vec • “Sentences” are extracted by performing random graph walks: Year Zero Nine Inch Nails Trent Reznor • Experiments – RDF2vec can be trained on large KGs (DBpedia, Wikidata) – 300-500 dimensional vectors outperform other propositionalization strategies artist member Ristoski & Paulheim: RDF2vec: RDF Graph Embeddings for Data Mining. ISWC, 2016
  • 19.
    7/2/19 Heiko Paulheim19 From word2vec to RDF2vec • RDF2vec example – similar instances form clusters – direction of relations is stable Ristoski & Paulheim: RDF2vec: RDF Graph Embeddings for Data Mining. ISWC, 2016
  • 20.
    7/2/19 Heiko Paulheim20 From word2vec to RDF2vec • RecSys example: using proximity in latent RDF2vec feature space Ristoski et al.: RDF2Vec: RDF Graph Embeddings and their Applications. SWJ 10(4), 2019
  • 21.
    7/2/19 Heiko Paulheim21 Extensions of RDF2vec • Maybe random walks are not such a good idea – They may give too much weight on less-known entities and facts ● Strategies: – Prefer edges with more frequent predicates – Prefer nodes with higher indegree – Prefer nodes with higher PageRank – … – They may cover less-known entities and facts too little ● Strategies: – The opposite of all of the above strategies • Bottom line of experimental evaluation: – Not one strategy fits all Cochez et al.: Biased Graph Walks for RDF Graph Embeddings. WIMS, 2017
  • 22.
    7/2/19 Heiko Paulheim22 Other Word Embedding Methods • GloVe (Global Word Embedding Vectors) • Computes embeddings out of co-occurence statistics – Using matrix factorization • Has been applied to random RDF walks as well • Experimental evaluation: – In some cases, RDFGloVe outperforms RDF2vec https://www.kdnuggets.com/2018/04/implementing-deep-learning-methods-feature-engineering-text-data- glove.html Cochez et al.: Global RDF Vector Space Embeddings, ISWC, 2017
  • 23.
    7/2/19 Heiko Paulheim23 Other Word Embedding Methods • There is a lot of promising stuff not yet tried – e.g., biasing walks based on human factors – e.g., more recent word embedding methods such as ELMo and BERT https://www.nbcnews.com/feature/nbc-out/bert-ernie-are-gay-couple-sesame-street-writer-claims-n910701
  • 24.
    7/2/19 Heiko Paulheim24 TransE and its Descendants • In RDF2vec, relation preservation is a by-product • TransE: direct modeling – Formulates RDF embedding as an optimization problem – Find mapping of entities and relations to Rn so that ● across all triples <s,p,o> Σ ||s+p-o|| is minimized ● try to obtain a smaller error for existing triples than for non-existing ones Bordes et al: Translating Embeddings for Modeling Multi-relational Data. NIPS 2013. Fan et al.: Learning Embedding Representations for Knowledge Inference on Imperfect and Incomplete Repositories. WI 2016
  • 25.
    7/2/19 Heiko Paulheim25 Limitations of TransE • Symmetric properties – we have to minimize ||Barack + spouse – Michelle|| and ||Michelle + spouse – Barack|| simultaneously – ideally, Barack + spouse = Michelle and Michelle + spouse = Barack ● Michelle and Barack become infinitely close ● spouse becomes 0 vector Michelle Barack
  • 26.
    7/2/19 Heiko Paulheim26 Limitations of TransE • Transitive Properties – we have to minimize ||Miami + partOf – Florida|| and ||Florida + partOf – USA||, but also ||Miami + partOf – USA|| – ideally, Miami + partOf = Florida, Florida + partOf = USA, Miami + partOf = USA ● Again: all three become infinitely close ● partOf becomes 0 vector Florida Miami USA
  • 27.
    7/2/19 Heiko Paulheim27 Limitations of TransE • One to many properties – we have to minimize ||New York + partOf – USA||, ||Florida + partOf – USA||, ||Ohio + partOf – USA||, … – ideally, NewYork + partOf = USA, Florida + partOf = USA, Ohio + partOf = USA ● all the subjects become infinitely close Florida USA New York Ohio
  • 28.
    7/2/19 Heiko Paulheim28 Limitations of TransE • Reflexive properties – we have to minimize ||Tom + knows - Tom|| – ideally, Tom + knows = Tom ● Knows becomes 0 vector Tom
  • 29.
    7/2/19 Heiko Paulheim29 TransE RDF2Vec HolE DistMult RESCAL NTN TransR TransH TransD KG2E ComplEx Limitations of TransE • Numerous variants of TransE have been proposed to overcome limitations (e.g., TransH, TransR, TransD, …) • Plus: embedding approaches based on tensor factorization etc.
  • 30.
    7/2/19 Heiko Paulheim30 Are we Driving on the Wrong Side of the Road?
  • 31.
    7/2/19 Heiko Paulheim31 Are we Driving on the Wrong Side of the Road? • Original ideas: – Assign meaning to data – Allow for machine inference – Explain inference results to the user Berners-Lee et al: The Semantic Web. Scientific American, May 2001
  • 32.
    7/2/19 Heiko Paulheim32 Running Example: Recommender Systems • Content based recommender systems backed by Semantic Web data – (today: knowledge graphs) • Advantages – use rich background information about recommended items (for free) – justifications can be generated (e.g., you like movies by that director) https://lazyprogrammer.me/tutorial-on-collaborative-filtering-and-matrix-factorization-in-python/
  • 33.
    7/2/19 Heiko Paulheim33 The 2009 Semantic Web Layer Cake
  • 34.
    7/2/19 Heiko Paulheim34 The 2019 Semantic Web Layer Cake Embeddings
  • 35.
    7/2/19 Heiko Paulheim35 Towards Semantic Vector Space Embeddings cartoon superhero Ristoski et al.: RDF2Vec: RDF Graph Embeddings and their Applications. SWJ 10(4), 2019
  • 36.
    7/2/19 Heiko Paulheim36 The Holy Grail • Combine semantics and embeddings – e.g., directly create meaningful dimensions – e.g., learn interpretation of dimensions a posteriori – ...
  • 37.
    7/2/19 Heiko Paulheim37 A New Design Space quantitative performance semantic interpretability
  • 38.
    7/2/19 Heiko Paulheim38 Software to Check Out • http://openke.thunlp.org/ – Implements many embedding approaches – Pre-trained vectors available, e.g., for Wikidata
  • 39.
    7/2/19 Heiko Paulheim39 Software to Check Out • Loading RDF in Python: https://github.com/RDFLib/rdflib
  • 40.
    7/2/19 Heiko Paulheim40 RapidMiner Linked Open Data Extension caution: works only until RM6! :-(
  • 41.
    7/2/19 Heiko Paulheim41 References (1) • Berners-Lee, T., Hendler, J., & Lassila, O. (2001). The semantic web. Scientific american, 284(5), 28-37. • Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., & Yakhnenko, O. (2013). Translating embeddings for modeling multi-relational data. In NIPS (pp. 2787-2795). • Cochez, M., Ristoski, P., Ponzetto, S. P., & Paulheim, H. (2017). Biased graph walks for RDF graph embeddings. In WIMS (p. 21). ACM. • Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. • Melo, A., Völker, J., & Paulheim, H. (2017). Type prediction in noisy RDF knowledge bases using hierarchical multilabel classification with graph and latent features. IJAIT, 26(02). • Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. • Paulheim, H., & Fümkranz, J. (2012). Unsupervised generation of data mining features from linked open data. In WIMS (p. 31). ACM. • Paulheim, H., & Bizer, C. (2013). Type inference on noisy RDF data. In International semantic web conference (pp. 510-525). Springer, Berlin, Heidelberg.
  • 42.
    7/2/19 Heiko Paulheim42 References (2) • Paulheim, H., & Bizer, C. (2014). Improving the quality of linked data using statistical distributions. IJSWIS, 10(2), 63-86. • Paulheim, H. (2017). Knowledge graph refinement: A survey of approaches and evaluation methods. Semantic web, 8(3), 489-508. • Paulheim, H. (2018). Make Embeddings Semantic Again! ISWC (Blue Sky Track) • Peters, M. E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018). Deep contextualized word representations. arXiv preprint arXiv:1802.05365. • Ristoski, P., & Paulheim, H. (2014). A comparison of propositionalization strategies for creating features from linked open data. Linked Data for Knowledge Discovery, 6. • Ristoski, P., Bizer, C., & Paulheim, H. (2015). Mining the web of linked data with rapidminer. Web Semantics: Science, Services and Agents on the World Wide Web, 35, 142-151. • Ristoski, P., & Paulheim, H. (2016). Semantic Web in data mining and knowledge discovery: A comprehensive survey. Web semantics, 36, 1-22. • Ristoski, P., & Paulheim, H. (2016). RDF2vec: RDF graph embeddings for data mining. In International Semantic Web Conference (pp. 498-514). Springer, Cham. • Ristoski, P., Rosati, J., Di Noia, T., De Leone, R., & Paulheim, H. (2019). RDF2Vec: RDF graph embeddings and their applications. Semantic Web, 10(4), 1-32.
  • 43.
    7/2/19 Heiko Paulheim43 Machine Learning & Embeddings for Large Knowledge Graphs Heiko Paulheim