A Java implemention of LDA(Latent Dirichlet Allocation). Inference topics from a set of documents with few lines of Java code.
- code
public static void main(String[] args) { // 1. Load corpus from disk Corpus corpus = Corpus.load("data/mini"); // 2. Create a LDA sampler LdaGibbsSampler ldaGibbsSampler = new LdaGibbsSampler(corpus.getDocument(), corpus.getVocabularySize()); // 3. Train it ldaGibbsSampler.gibbs(10); // 4. The phi matrix is a LDA model, you can use LdaUtil to explain it. double[][] phi = ldaGibbsSampler.getPhi(); Map<String, Double>[] topicMap = LdaUtil.translate(phi, corpus.getVocabulary(), 10); LdaUtil.explain(topicMap); }- output
topic 0 : 公司=0.009538408630174017 市场=0.008848009751698062 中国=0.008756489189917975 企业=0.0068280510303913395 发展=0.005991900977658479 目前=0.004408401842957633 产品=0.0041981128106208625 服务=0.003756081561227181 已经=0.003410105744626914 记者=0.003289155629929911 topic 1 : 专业=0.00872496522205349 工作=0.008108171408190876 学生=0.00793944661866665 学校=0.006307480899983371 考生=0.005295205701518912 大学=0.0052671267600129445 教育=0.0051547106121291805 考试=0.00507254577329609 人才=0.004037747449851247 招聘=0.003913811857165103 topic 2 : 医院=0.006197066939127888 治疗=0.0048149451145789455 患者=0.0032264139617756145 健康=0.0026521203697810374 手术=0.0025525793863978826 女性=0.0023724111474892357 专家=0.0021711200905248276 发现=0.0021645199996586885 病人=0.0021567877663232846 医生=0.002155356316589454 topic 3 : 没有=0.008818728535385055 问题=0.00476170232225101 中国=0.00476161560515722 工作=0.004610303190509696 生活=0.004283310385880329 文化=0.0036558079614339278 孩子=0.003327977201447208 不能=0.0032901108349775716 知道=0.003127437274214269 已经=0.0030419673256694545 topic 4 : 公司=0.018241005428669386 股东=0.009281048036676322 股份=0.0078638937643388 搜狐=0.0065617441267974705 有限公司=0.006139808167975946 直播员=0.005439495997416965 股权=0.005353954615162839 项目=0.004984451830097043 发行=0.004511099443364358 改革=0.004489038403046334 topic 5 : 旅游=0.013331508385667979 游客=0.004296589238778804 城市=0.0032312276892446116 文化=0.0026831367778820704 旅行社=0.002242817493567529 世界=0.0021001546909288965 成都=0.001991337289815279 活动=0.001894687770595843 北京=0.0017106388886854072 公园=0.0016134766410937638 topic 6 : 美国=0.007679518424242107 日本=0.004777746687572576 训练=0.003947682941243526 系统=0.003926562149803556 飞机=0.0038757503504304267 部队=0.00365041154980242 进行=0.003644226666909795 军事=0.003637873811678725 作战=0.003407296869780034 装备=0.003319112427162246 topic 7 : 比赛=0.0092171879571152 队员=0.0036851386114063237 联赛=0.0032845199043377146 球队=0.0029432131822116707 冠军=0.0024090127058022104 俱乐部=0.002348957542679953 球员=0.0022159606741087795 决赛=0.002192739194333911 赛季=0.0020352324832133267 对手=0.001974829226645783 topic 8 : The=0.002190604616155811 意思=0.001186435720799536 It=0.0011515962078723501 理解=0.0010433831740419728 What=9.560997173453189E-4 They=9.345962358267594E-4 听力=8.362275772461826E-4 In=8.166984660263638E-4 阅读=7.775969918239417E-4 译文=7.568900132152651E-4 topic 9 : 毛泽东=0.002633793448326645 曹操=0.0018832599387516155 曹丕=0.0016567353952110328 皇帝=0.001629990508040292 甄洛=0.0012930147890964736 中央=0.0012783947883529055 蒋介石=0.0010732052837016102 曹睿=8.511476483731437E-4 女王=8.125680914406854E-4 皇后=8.013815303127338E-4 - corpus The data/mini is some documents included in this project, which use space to segment words. Feel free to replace it with yours.
- algorithm Mainly depend on Gregor Heinrich's great work. Read more about this implementation on《LDA入门与Java实现》