Skip to content

Latest commit

 

History

History
370 lines (286 loc) · 12.4 KB

Upgrade.md

File metadata and controls

370 lines (286 loc) · 12.4 KB
Raw

Upgrading to support a new version of GSL

The travis build script

When upgrading to a new version, the travis CI builds will tell us when something is not right. Currently we are building and testing for 58 different setups (combinations of GSL version and perl version).

Stage 1

For each setup, the build script first builds a CPAN distribution Math-GSL-yy.xx.tar.gz where yy.xx is the current version of the the swig module, i.e. the same as $MATH::GSL::VERSION.

First the GSL version given by the GSL_CURRENT environment variable, see .travis.yml, is downloaded and installed.

Then, perl Build.PL is run. It uses gsl-config to determine the installed version of GSL on the system. Here you can change the PATH environment variable before calling perl Build.PL in order for it to use another version of GSL if you have multiple versions installed. In our case it determines that the GSL version given by GSL_CURRENT is installed.

Then, Build.PL extracts the GSL version by running gsl-config and writes this information to swig/system.i. Finally, it generates a script called Build.

Next, the Build script is run, which will iterate through all the versions as given in the @ver2func array in inc/Ver2Func.pm. For each version, it first generates a rename.i file in the swig/ directory based on the version in @ver2func and then runs the swig command on all interface files found in the swig/ directory that also matches the current subsystem information found in @ver2func for the current version.

The output of each swig call is a .pm file which is put in the pm/ directory, and a .c file which is put in the xs/ directory.

After Build is finished, travis calls Build dist to generate a CPAN distribution archive file. This will include all the files listed in the MANIFEST file.

NOTE:

  • No tests (the test files are in the t/ directory) are run at this stage.
  • The generated distribution is independet of the swig program, so the distribution can be installed on a target machine that does not have swig installed.

Stage 2

In this stage, the travis build script tries to build and test the CPAN distribution generated in stage 1 above for the given system (one of 58 setups described above). So this stage is independent of the swig binary.

First, if the GSL version given by $GSL is not installed, it will download and install it.

Then it extracts the distribution in a separate directory, runs perl Build.PL, ./Build, and finally ./Build test runs the tests as given in the /t directory.

What to do?

In order to upgrade to a new version of GSL:

Update MANIFEST

Update the MANIFEST files to include the .pm and .c files that will be generated by running swig from ./Build. It is usually not necessary to run Perl Build.pl; Build to determine the file names. You can simply look at the files currently in the MANIFEST file and deduce what the new names will be.

Update .tavis.yml

  • First, update the GSL_CURRENT item in global: under the env: list to reflect the new version of GSL.

  • Add the previous (i.e., the current before we did this upgrade) to the head of the matrix: list under the env: list.

  • Consider removing unsupported GSL versions from the end of the list.

  • Consider adding new versions of perl to the front of the perl: list.

  • Consider deleting deprecated versions of perl from the end of the perl: list.

Update Ver2Func.pm

This is the main part of the upgrading work, and it can be time consuming depending on the number of new features that has been added.

First, add a new hash entry to the end of the @ver2func array. Now, all functions that are new to this version compared to the previous version should be added in the new item of the hash. How to determine which items are new? [TODO: find a simpler way to do this?] The brute force approach is to run the test suite repeatedly looking at the failed tests will given an indication of the names of the new functions.

When all the tests passes, we can be confident that we have added most of the new functions.

There can also be C structures that have been rewritten in the new version compared to the old such that the names/fields in the structure are different. Since swig generates accessor functions for the structures it will typically also lead to failed tests.

An example of a difficult case

When upgrading from 2.5 to 2.6: For GSL 2.5 the header gsl_spmatrix.h looked like this (some comments removed for brevity):

/* gsl_spmatrix.h
 *
 * Copyright (C) 2012-2014 Patrick Alken
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 3 of the License, or (at
 * your option) any later version.
 *
 * This program is distributed in the hope that it will be useful, but
 * WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
 * General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program; if not, write to the Free Software
 * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
 */

#ifndef __GSL_SPMATRIX_H__
#define __GSL_SPMATRIX_H__

#include <stdlib.h>

#include <gsl/gsl_math.h>
#include <gsl/gsl_vector.h>
#include <gsl/gsl_matrix.h>

#undef __BEGIN_DECLS
#undef __END_DECLS
#ifdef __cplusplus
# define __BEGIN_DECLS extern "C" {
# define __END_DECLS }
#else
# define __BEGIN_DECLS /* empty */
# define __END_DECLS /* empty */
#endif

__BEGIN_DECLS

/*
 * Binary tree data structure for storing sparse matrix elements
 * in triplet format. This is used for efficiently detecting
 * duplicates and element retrieval via gsl_spmatrix_get
 */
typedef struct
{
  void *tree;       /* tree structure */
  void *node_array; /* preallocated array of tree nodes */
  size_t n;         /* number of tree nodes in use (<= nzmax) */
} gsl_spmatrix_tree;

typedef struct
{
  size_t size1;  /* number of rows */
  size_t size2;  /* number of columns */

  size_t *i;

  double *data;  /* matrix elements of size nzmax */
  size_t *p;

  size_t nzmax;  /* maximum number of matrix elements */
  size_t nz;     /* number of non-zero values in matrix */

  gsl_spmatrix_tree *tree_data; /* binary tree for sorting triplet data */

  union
    {
      void *work;
      size_t *work_sze;
      double *work_dbl;
    };

  size_t sptype; /* sparse storage type */
} gsl_spmatrix;

#define GSL_SPMATRIX_TRIPLET      (0)
#define GSL_SPMATRIX_CCS          (1)
#define GSL_SPMATRIX_CRS          (2)

#define GSL_SPMATRIX_ISTRIPLET(m) ((m)->sptype == GSL_SPMATRIX_TRIPLET)
#define GSL_SPMATRIX_ISCCS(m)     ((m)->sptype == GSL_SPMATRIX_CCS)
#define GSL_SPMATRIX_ISCRS(m)     ((m)->sptype == GSL_SPMATRIX_CRS)

gsl_spmatrix *gsl_spmatrix_alloc(const size_t n1, const size_t n2);
gsl_spmatrix *gsl_spmatrix_alloc_nzmax(const size_t n1, const size_t n2,
                                       const size_t nzmax, const size_t flags);
void gsl_spmatrix_free(gsl_spmatrix *m);
int gsl_spmatrix_realloc(const size_t nzmax, gsl_spmatrix *m);
int gsl_spmatrix_set_zero(gsl_spmatrix *m);
size_t gsl_spmatrix_nnz(const gsl_spmatrix *m);
int gsl_spmatrix_compare_idx(const size_t ia, const size_t ja,
                             const size_t ib, const size_t jb);
int gsl_spmatrix_tree_rebuild(gsl_spmatrix * m);

int gsl_spmatrix_memcpy(gsl_spmatrix *dest, const gsl_spmatrix *src);

double gsl_spmatrix_get(const gsl_spmatrix *m, const size_t i,
                        const size_t j);
int gsl_spmatrix_set(gsl_spmatrix *m, const size_t i, const size_t j,
                     const double x);
double *gsl_spmatrix_ptr(gsl_spmatrix *m, const size_t i, const size_t j);

gsl_spmatrix *gsl_spmatrix_compcol(const gsl_spmatrix *T);
gsl_spmatrix *gsl_spmatrix_ccs(const gsl_spmatrix *T);
gsl_spmatrix *gsl_spmatrix_crs(const gsl_spmatrix *T);
void gsl_spmatrix_cumsum(const size_t n, size_t *c);

int gsl_spmatrix_fprintf(FILE *stream, const gsl_spmatrix *m,
                         const char *format);
gsl_spmatrix * gsl_spmatrix_fscanf(FILE *stream);
int gsl_spmatrix_fwrite(FILE *stream, const gsl_spmatrix *m);
int gsl_spmatrix_fread(FILE *stream, gsl_spmatrix *m);

int gsl_spmatrix_scale(gsl_spmatrix *m, const double x);
int gsl_spmatrix_minmax(const gsl_spmatrix *m, double *min_out,
                        double *max_out);
int gsl_spmatrix_add(gsl_spmatrix *c, const gsl_spmatrix *a,
                     const gsl_spmatrix *b);
int gsl_spmatrix_d2sp(gsl_spmatrix *S, const gsl_matrix *A);
int gsl_spmatrix_sp2d(gsl_matrix *A, const gsl_spmatrix *S);
int gsl_spmatrix_equal(const gsl_spmatrix *a, const gsl_spmatrix *b);
int gsl_spmatrix_transpose(gsl_spmatrix * m);
int gsl_spmatrix_transpose2(gsl_spmatrix * m);
int gsl_spmatrix_transpose_memcpy(gsl_spmatrix *dest, const gsl_spmatrix *src);

__END_DECLS

#endif /* __GSL_SPMATRIX_H__ */

whereas in GSL 2.6, gsl_spmatrix.h looked like this:

#ifndef __GSL_SPMATRIX_H__
#define __GSL_SPMATRIX_H__

enum
{
  GSL_SPMATRIX_COO = 0, /* coordinate/triplet representation */
  GSL_SPMATRIX_CSC = 1, /* compressed sparse column */
  GSL_SPMATRIX_CSR = 2, /* compressed sparse row */
  GSL_SPMATRIX_TRIPLET = GSL_SPMATRIX_COO,
  GSL_SPMATRIX_CCS = GSL_SPMATRIX_CSC,
  GSL_SPMATRIX_CRS = GSL_SPMATRIX_CSR
};

struct gsl_spmatrix_pool_node
{
  struct gsl_spmatrix_pool_node * next;
  void * block_ptr;          /* pointer to memory block, of size n*tree_node_size */
  unsigned char * free_slot; /* pointer to next available slot */
};

typedef struct gsl_spmatrix_pool_node gsl_spmatrix_pool;

#define GSL_SPMATRIX_ISCOO(m)         ((m)->sptype == GSL_SPMATRIX_COO)
#define GSL_SPMATRIX_ISCSC(m)         ((m)->sptype == GSL_SPMATRIX_CSC)
#define GSL_SPMATRIX_ISCSR(m)         ((m)->sptype == GSL_SPMATRIX_CSR)

#define GSL_SPMATRIX_ISTRIPLET(m)     GSL_SPMATRIX_ISCOO(m)
#define GSL_SPMATRIX_ISCCS(m)         GSL_SPMATRIX_ISCSC(m)
#define GSL_SPMATRIX_ISCRS(m)         GSL_SPMATRIX_ISCSR(m)

#define GSL_SPMATRIX_FLG_GROW         (1 << 0) /* allow size of matrix to grow as elements are added */
#define GSL_SPMATRIX_FLG_FIXED        (1 << 1) /* sparsity pattern is fixed */

#define GSL_SPMATRIX_COMPARE_ROWCOL(m,ia,ja,ib,jb)   ((ia) < (ib) ? -1 : ((ia) > (ib) ? 1 : ((ja) < (jb) ? -1 : ((ja) > (jb)))))


void gsl_spmatrix_cumsum(const size_t n, int * c);

#include <gsl/gsl_spmatrix_complex_long_double.h>
#include <gsl/gsl_spmatrix_complex_double.h>
#include <gsl/gsl_spmatrix_complex_float.h>

#include <gsl/gsl_spmatrix_long_double.h>
#include <gsl/gsl_spmatrix_double.h>
#include <gsl/gsl_spmatrix_float.h>

#include <gsl/gsl_spmatrix_ulong.h>
#include <gsl/gsl_spmatrix_long.h>

#include <gsl/gsl_spmatrix_uint.h>
#include <gsl/gsl_spmatrix_int.h>

#include <gsl/gsl_spmatrix_ushort.h>
#include <gsl/gsl_spmatrix_short.h>

#include <gsl/gsl_spmatrix_uchar.h>
#include <gsl/gsl_spmatrix_char.h>

#endif /* __GSL_SPMATRIX_H__ */

so it had been completely refactored. This problem was solved in an ad-hoc way be rewriting Spmatrix.i from (GSL 2.5)

%module "Math::GSL::SparseMatrix"

%include "typemaps.i"
%include "gsl_typemaps.i"
%include "renames.i"
%include "system.i"
%{
    #include "gsl/gsl_spmatrix.h"
%}
%include "gsl/gsl_spmatrix.h"
%include "../pod/SparseMatrix.pod"

to (GSL 2.6) :

%module "Math::GSL::SparseMatrix"
%include "typemaps.i"
%include "gsl_typemaps.i"
%include "renames.i"
%include "system.i"
%{
    #include "gsl/gsl_spmatrix.h"
%}
#if MG_GSL_NUM_VERSION >= 002006
  %include "gsl/gsl_spmatrix.h"
  %include "gsl/gsl_spmatrix_double.h"
  %include "gsl/gsl_spmatrix_complex_long_double.h"
  %include "gsl/gsl_spmatrix_complex_float.h"
  %include "gsl/gsl_spmatrix_long_double.h"
  %include "gsl/gsl_spmatrix_uint.h"
  %include "gsl/gsl_spmatrix_double.h"
  %include "gsl/gsl_spmatrix_complex_double.h"
  %include "gsl/gsl_spmatrix_char.h"
  %include "gsl/gsl_spmatrix_uchar.h"
  %include "gsl/gsl_spmatrix_int.h"
  %include "gsl/gsl_spmatrix_short.h"
  %include "gsl/gsl_spmatrix_float.h"
  %include "gsl/gsl_spmatrix_ushort.h"
#else
  %include "legacy/gsl-2.5/gsl_spmatrix.h"
#endif
%include "../pod/SparseMatrix.pod"

So for this complicated case we skipped the rename.i approach where we just add new function names to @ver2func in Ver2Func.pm as described above, and simply included the whole gsl_spmatrix.h header file in the local include/legacy directory.