next up previous
Next: 2.5 Structure Learning Up: 2 Notes on Code Previous: 2.3 Logic


2.4 Weight Learning

The learnwts directory contains code for learning the weights of formulas. The mainline is in learnwts.cpp. Table 1 shows the options available when calling learnwts.


<-i <string» Comma-separated input .mln files. (With the -multipleDatabases option, the second file to the last one are used to contain constants from different databases, and they correspond to the .db files specified with the -t option.)
[-cw <string>] Specified non-evidence atoms (comma-separated with no space) are closed world, otherwise, all non-evidence atoms are open world. Atoms appearing here cannot be query atoms and cannot appear in the -o option.
[-ow <string>] Specified evidence atoms (comma-separated with no space) are open world, while other evidence atoms are closed-world. Atoms appearing here cannot appear in the -c option.
[-infer <string>] Specified inference parameters when using discriminative learning. The arguments are to be encapsulated in "" and the syntax is identical to the infer command (run infer with no commands to see this). If not specified, MaxWalkSat with default parameters is used.
[-d [bool]] Discriminative weight learning.
[-g [bool]] Generative weight learning.
<-o <string» Output .mln file containing formulas with learned weights.
<-t <string» Comma-separated .db files containing the training database (of true/false ground atoms), including function definitions, e.g. ai.db,graphics.db,languages.db.
[-ne <string>] First-order non-evidence predicates (comma-separated with no space), e.g., cancer,smokes,friends. For discriminative learning, at least one non-evidence predicate must be specified. For generative learning, the specified predicates are included in the (weighted) pseudo-log-likelihood computation; if none are specified, all are included.
[-noAddUnitClauses [bool]] If specified, unit clauses are not included in the .mln file; otherwise they are included.
[-multipleDatabases [bool]] If specified, each .db file belongs to a separate database; otherwise all .db files belong to the same database.
[-withEM [bool]] If set, EM is used to fill in missing truth values; otherwise missing truth values are set to false.
[-dNumIter <integer>] [200] (For discriminative learning only.) Number of iterations to run voted perceptron.
[-dLearningRate <double>] [0.001] (For discriminative learning only) Learning rate for the gradient descent in voted perceptron algorithm.
[-dMomentum <double>] [0.0] (For discriminative learning only) Momentum term for the gradient descent in voted perceptron algorithm.



Table 1: Command line options for learnwts
[-queryEvidence [bool]] If this flag is set, then all the groundings of query preds not in db are assumed false evidence.
[-dRescale [bool]] (For discriminative learning only.) Rescale the gradient by the number of true groundings per weight.
[-dZeroInit [bool]] (For discriminative learning only.) Initialize clause weights to zero instead of their log odds.
[-gMaxIter <integer>] [10000] (For generative learning only.) Max number of iterations to run L-BFGS-B, the optimization algorithm for generative learning.
[-gConvThresh <double>] [1e-5] (For generative learning only.) Fractional change in pseudo-log-likelihood at which L-BFGS-B terminates.
[-gNoEqualPredWt [bool]] (For generative learning only.) If specified, the predicates are not weighted equally in the pseudo-log-likelihood computation; otherwise they are.
[-noPrior [bool]] No Gaussian priors on formula weights.
[-priorMean <double>] [0] Means of Gaussian priors on formula weights. By default, for each formula, it is the weight given in the .mln input file, or fraction thereof if the formula turns into multiple clauses. This mean applies if no weight is given in the .mln file.
[-priorStdDev <double>] [1 for discriminative learning. 100 for generative learning] Standard deviations of Gaussian priors on clause weights.


If you do not want to print the clauses as their number of true groundings are being counted during generative learning, you can set the variable PRINT_CLAUSE_DURING_COUNT to false at the top of learnwts.cpp. learnwts.h contains functions used in learnwts.cpp that can be shared with other modules. PseudoLogLikelihood computes the (weighted) pseudo-log-likelihood given the constants in one or more Domains, and clauses in an MLN. LBFGSB is an optimization routine that finds the optimal weights, i.e., the weights that give the highest (weighted) pseudo-log-likelihood. VotedPerceptron contains the algorithm for discriminative learning. IndexTranslator is used to translate between clause weights and the weights that are optimized. It is required when the CNF of a formula is different across multiple databases, e.g., when the formula has existentially quantified variables, or variables with mutually exclusive and exhaustive values.


next up previous
Next: 2.5 Structure Learning Up: 2 Notes on Code Previous: 2.3 Logic
Marc Sumner 2007-01-16