Historic_ChangeLog

Version 18

	1)	Switched build process to use Apache Maven. Changed directory structure. Removed ant and Make files. Most jar files are now pulled directly from the Maven repository.
	2)	Upgraded Java version as Java 8 will be going out of support. Artemis will now only build/run with Java versions 9 to 11 (RT ticket #589103).
	3)	Upgraded to Picard 2.18.14 jar, which has a number of important bug fixes, including:
		i)  RT tickets #625730 - Loading bams into Artemis
		ii) RT ticket #624271 -  Problems with some bams loaded into Artemis causing data not to be viewable
	4)	Removed broken documentation functionality. 
		HTML/PDF manuals are now stored in Github gh-pages branch and exported from word docs (RT ticket #618907).
	5)	Added run menu option to execute an InterPro sequence search for a selected feature - RT ticket #643820.
	6)	Updated application scripts.
	7)	Fixed broken Amigo, GO term, Uniprot (RT ticket #639930) and UniprotKB URLs, used in the Artemis gene builder/feature edit windows. 
	8)	Updated README.md
	9)	Fix for RT ticket #400288: "GO term warnings in Artemis" - we now no longer put out a warning for ISM, as it's optional in certain cases
	    [http://wiki.geneontology.org/index.php/Inferred_from_Sequence_Model_(ISM)].
	10)	Removed the INSTALL document as the content is now covered elsewhere.
	11)	Changed error message displayed when multiple features are selected for pfam/rfam search.
	12)	Added temporary Java 9+ --add-opens flags to art/act scripts. These will be removed when Ibatis is replaced.
	13)	Added unit tests for Chado access to enable Postgres 10 testing (RT ticket #632896).
	14)	Added Mockito for unit test mocking.
	15)	Updated/fixed run_blast<..> scripts in the etc folder. Running blast from the run menu now uses blast+. Removed options to run Smart, Jalview, fasta & fastx.
		Results are displayed using View->Search Results. 
		Related RT ticket #596281 also closed.
	16)	Removed the old icons/Readme doc for local blast setup and added an updated version to the main README.md.
	17)	Updated setup_blast_dbs.sh script and added to etc folder, for setup of a local blast environment (detailed in the README.md and manuals).
	18)	Removed ability to run remote blast, fasta34 etc using SSH. This functionality is being retired.
	19)	Removed old ".sanger" and ".sanger.linux" scripts from etc folder - these were very out of date, with incorrect file paths etc.
	20)	Fixed RT ticket #642350: "GO term evidence codes in chado Artemis" - Added HTP, HDA, HMP, HGI, HEP GO evidence codes to Artemis.
	21)	Added fix for Java 9 JDK-8181568 issue across all apps, related to display of application icons on Macs.
	22)	Fixed incorrect Pfam URL, used in Feature Edit and Evidence Viewer screens.
	23)	Fixed error handling on Run->Pfam/Rfam search (RT ticket #420623).
	24)	Enabled ability to drag and drop data files on to Artemis and BamView apps [Mac OS X] for immediate display.
	25)	Fixed RT ticket #642944: "ACT crunch-m8 file generation". ACT can now handle tblastx hit table files produced from comparisons 
		on the NCBI Blast web site. This is an alternative to Web ACT / Double ACT.
	    [e.g. https://blast.ncbi.nlm.nih.gov/Blast.cgi?PROGRAM=tblastx&PAGE_TYPE=BlastSearch&BLAST_SPEC=blast2seq&LINK_LOC=blasttab&LAST_PAGE=tblastx&BLAST_INIT=blast2seq]
	26)	Removed the /lustre/scratch101/blastdb/Pathogen/Kineto_aa run->blast menu option. This database path no longer exists.
	27)	RT ticket #472601: Artemis RPKM and read count broken since version 16.0.6. Fixed bug related to intron region reads. New unit tests added.
	
	
Version 17.0.2-test - patch release for Sanger. Changes incorporated into GitHub master.
 
 	1) Added bamview_use_htsjdk_file_index_caching property to allow switching off of htsjdk caching for bam files. 
 	   This gets around a bug for large bams whereby parts of the bam fail to get loaded - RT ticket #624271. 
 	   See htsjdk GitHub issue 1127 for details.
 
Version 17

	Artemis/ACT and associated applications have been upgraded to Java 8. Java 7/6/5 are now no longer supported - RT ticket #589103
	
	BamView fixes/changes - 
		1) Improved mouse selection of reads.
		2) Fixed exceptions thrown when zooming into an area with secondary reads or SNPs enabled - RT ticket #596489: Problem zooming in using isoseq data
		3) Removed Samtools (htsjdk) jar and updated Picard jar (which includes htsjdk).
		4) Upgraded Apache Commons Net jar - used for ftp'ing. 
		5) Added Cram file loading and fixed defects around that code area (also related RT ticket #560982: Artemis fails to build from source).
		6) Added display of bam secondary & supplementary alignment flags to read details pop-up window.
		7) Improved error handling/reporting, although this still needs more work.
		8) Added buffer sizing to FTPSeekableStream.
		9) Performance tweak for BAM file indexing.
		10) Fixed issue in BamView whereby if you went to the end of a long contig and then switched to a shorter one (via the combo), you would end up off the end of the sequence
			resulting in a negative array index exception. Code has been changed to reset display to the start of sequence when the combo is changed (i.e. same as Artemis).
		11) Added bamview_perform_detailed_validation boolean property that can be set to perform more detailed sam validation during loading - this can be slow for large files.
		
	Added EvoSuite unit tests and additional targeted tests (RT ticket #419534: Develop a test suite for Artemis).
	
	Added Jacoco unit test coverage reporting - manual tests only currently. This may be switched to Clover in the future if this proves compatible with EvoSuite
		- for the moment we are stuck with two coverage reports - evosuite and Jacoco.
	
	Removed redundant Corba libraries.
	
	Upgraded JUnit jar.
	
	Changed Travis yml build file. RT ticket #597497: Set up artemis tests to run on travis build.
	
	Changed all build scripts and startup scripts - RT ticket #598617. Removed building of artemis_mac.jar which is not used. 
	Removed etc/gene_builder script and the gff2embl script (and documentation references) - no longer supported. 
	
	Fixed RT ticket #606061 - Using EBI-Dbfetch on ARTEMIS. Change to DbfetchEntrySource.java due to incorrect regular expression and EBI URL.
	
	Resized splash screen, as it was too small to the accommodate text info.
	
	Updated splash screen logo.
	
	Added error handling for dnaplotter template file loading in standalone mode.
	
	Bioconda recipe prep work - RT ticket #341139.
	
	Fixed RT ticket #467433 - Genbank DBSOURCE field was not recognised by Artemis. DBLINK was already added.
	
	Fixed RT ticket #503254 - Act doesn't detect if file does not exist.
	
	KNOWN ISSUES: 
		1) Java JDKs 1.8.0_131 and above have a Swing bug related to overlaying of modal dialogs for Mac OS X: https://bugs.openjdk.java.net/browse/JDK-8179335
		2) There is a current bug in htsjdk whereby calls to the queryMate functionality can throw an exception on reads with secondaries and/or supplementals. 
		   This affects looking up properties for a selected read on bamview - https://github.com/samtools/htsjdk/issues/1065

Version 16
	Add 'Features Within Selection' option to the 'Select' menu to select
	features that are contained by a selected base range.

	Saving an entry as EMBL submission has an option now to remove products
	from CDS features with a pseudogene qualifier.
	
	From the 'View' menu added 'Adjust panel heights...' option in ACT 
	for adjusting their heights (BAM, VCF, plots, comparisons) by giving 
	them different weights in order to distribute the space between each 
	component.
	
	Added ability to hide graph lines. Right click on the graph to get the 
	popup menu and select 'Configure...'. Then use the 'Line size' slider 
	to reduce the size of the line(s) that you want to hide to zero.

	Labels can optionally be added to the header of base position plots
	and these are used in the legend, e.g.
	# colour 5:150:55 255:0:0 0:255:0 0:0:255 100:100:100 50:150:50
	# label lab1 lab2 lab3 lab4 lab5 lab6
	176 2204.8 848.23 0 0 0 536.04
	...
	
	In the Feature Editor there is now a button ('User Qualifiers') which
	opens a tool for maintaining user defined lists of qualifiers (i.e. 
	qualifiers in the form tag = value pairs on separate lines) and the 
	option to read qualifiers from OBO formatted files or URLs. These can
	be saved between sessions in the file '.artemis.qualifiers' in the
	home directory.
	
	Automatic addition of history qualifier in chado database mode.

	Added RNASeq strand specific option to the BAM popup menu ('Colour By' -> 
	'RNASeq Strand Specific Tag (XS)'). Reads colours are based on the XS 
	tag (used by TopHat). The RNA strand is then used in the coverage plots
	and in calculation of read counts and RPKM values.
	
	Added the following flags:
  		-Dshow_snps			Show SNP marks in BamView
  		-Dshow_snp_plot		Open SNP plot in BamView
  		-Dshow_cov_plot		Open coverage plot in BamView
  
	Add validation checks. This will check the following:
		All file types:
	    	- CDS have no internal stop codon
      		- CDS have valid stop codon
		GFF / Chado:
	    	- check complete gene model
        	- check boundaries are valid
        	- check all features are on the same strand
        	- check CDS features have a phase
        	- check attribute column 
            - qualifiers have a value (not empty)
            - only reserved tags start with uppercase
	
		Validation can be run in two ways: 
			1. using the option in View->Feature Filters->Validation checks... which shows 
			'failed' features in feature list windows
			2. selecting 'Validation report ...' option in the popup menu when right clicking 
			on the feature display. This produces a report with an option to auto-fix gene 
			boundaries and stop codons. 

Version 15

	Multiple BAM panels can be opened using the bamClone flag this is used with 
	the -Dbam flag:
	art -Dbam='/pathToFile/file1.bam,/pathToFile/file2.bam' -DbamClone=n  
	(where n is an integer greater than 1). All BAM files are then shown in	each 
	panel. Alternatively the following will open BAM files in separate panels
	(using -Dbam[1,2,3...]):
	art -Dbam1=fileA.bam -Dbam2=fileB.bam
	
	Add SVG (scalable vector graphics) support for Artemis, ACT and DNAPlotter.

	Option added to adjust VCF row height.
		
	Add support for indexed user graphs using tabix. For example file.plot is a tab
	delimited file with column 1 containing the sequence name and column 2 the
	positions:
	(grep ^"#" file.plot; grep -v ^"#" file.plot | sort -k1,1 -k2,2n) | bgzip > sorted.plot.gz ;
	tabix -s 1 -b 2 -e 2 sorted.plot.gz
	
	Option added to show or hide the average line in the graphs.
	
	BAM coverage heatmap view added.

	Add Rfam sequence search from the RUN menu.
	
	Base similarity graph for each VCF added to the VCF view.
	
	Sense and anti-sense read counts and RPKM values are now reported.
	
	Option added to create features from BAM peaks, i.e. above a threshold of the 
	number aligned reads.

	Option included for defining groups of BAM files so that they can be
	switched on and off by their group.
	
	Coverage plots from read alignments (BAM) can be plotted by their strand.

	Addition of a Project File Manager used to group files together for
	launching in Artemis. When a project has been added or updated the details 
	are saved at the end of each session in '.artemis.project.properties' 
	in the user's home directory. 

	Fix for saving user defined shortcuts between sessions on windows.
	
	Add preliminary support for the CRAM format:
	http://www.ebi.ac.uk/ena/about/cram_toolkit
	This requires the cramtools.jar to be added to the start of the CLASSPATH.

	Improved support for matching GFF feature coordinates to the correct
	contig in a multiple FASTA sequences. This works now whether
	the sequence is part of the GFF file or a separate multiple-FASTA file.

	Add 'Feature Stack View' to visualise overlapping gene features.

	Add support for read-only indexed GFF. Features in GFF format are
	sorted and indexed with tabix:
	http://samtools.sourceforge.net/tabix.shtml

Version 14
	Add options in the navigator for searching the forward and 
	reverse strands individually for base / amino acid patterns.
	
	Add an option to the navigator for searching for matches that
	overlap a selected region or feature.
	
	New translation table 24 (Pterobranchia mitochondrial) added.
	
	If the BAM index file is missing then Artemis uses the picard library to
	attempt to create the index.

	Variant (VCF / BCF) filtering now uses the meta-data in the header to
	enable filtering based on INFO, FILTER and FORMAT columns.
	
	Shortcut changes made in the 'Preferences' menu are saved between sessions (the 
	shortcut_cache flag in the options file can be used to turn this on/off).
	
	More support for GTF format to show CDS and exons as joined features. 

	BAM record list option added to display as a list the reads and their
	properties.
	
	Add options for loading graph, BAM and VCF files into ACT from the 
	command line. Numbers are used to associate the file with a particular
	sequence. e.g. add a BAM to the first (top) sequence in ACT:
	act -Dbam1=/pathToFile/file.bam 
	or, to add a userplot to the second sequence:
	act -Duserplot2=/pathToFile/userPlot

Version 13.2.0
	Added option to provide overview of the variation sites.

	Change BamView filter to enable filtering in and out based on
	reads flag.
	
	Add read count and RPKM calculations to BamView.

	Add option in BamView to clone the alignment panel.
	
	Create features option for VCF records.
	
	Unit tests added for writing VCF/BCF sequences.

	Add new Coverage view to BamView. It automatically switches to this view
	on zooming out.

	Add options to write out or view FASTA sequences from VCF/BVF variation
	data.

Version 13
	Add support for reading in indexed BCF (Binary VCF) files.
	
	Memory optimisation of codon caching, reducing the memory footprint.
	
	It is now possible to add BAM, VCF and BCF files from the command line using the 
	JVM bam option, e.g. art -Dbam=/pathToFile/file.bam 
	and for multiple BAM's/VCF's this is comma separated
	art -Dbam='/pathToFile/file1.bam,/pathToFile/file2.bam', it
	can also read the BAM's from URL's.
	
	Added option to display orientation of reads in BamView.

	In Bamview, display reads that are split over introns so that the exon boundaries can 
	be identified by colouring the line between the aligned blocks grey.
	
	Add VCF panel to ACT.
	
	Support added to read indexed fasta sequence files. An index file is created
	using SAMtools.

	Add support for VCF v3.3 and v4.0.

	Add option to show combined coverage plots for multiple BAMs.

	Add show_forward_lines and show_reverse_lines as options for switching
	frame lines on and off.
	
	Add an option to delete qualifiers in the Find/Replace tool.

	Artemis can read in a set of zipped search results (e.g. blastp/blastp.zip).

	Add BamView panel to ACT.

	Fix for writing EMBL files out from GFF entries.

	An option has been added in BamView to colour reads by the colour 
	used in the coverage plot - useful when looking at multiple BAM files.
	
	Plot multiple coverage plots in BamView when multiple BAM files are
	loaded in.

Version 12
	It is now possible to add user plots from the command line using the 
	JVM userplot option, e.g. art -Duserplot=/pathToFile/userPlot 
	and for multiple plots this is comma separated
	art -Duserplot='/pathToFile/userPlot1,/pathToFile/userPlot2', it
	can also read the plots from URL's.

	Added support to be able to read BAM files to display read alignments.
	It uses picard (http://picard.sourceforge.net/) to read from the BAM file
	and so requires Artemis to be run with Java 1.6.

	All graphs can now be configured to be plotted as line graphs
	or heat maps.
	
	Added support for BLAST tabular format (-m 8 option) and MSPcrunch 
	format to plot scores.
	
	Added support for wiggle (variableStep/fixedStep) plots that can be 
	displayed as histograms or heat maps. 
	
	Added a new user plot file format with the first column specifying
	a base position.
	
	More configure options have been added to the graphs to enable
	configuration of the graph line style and size.
	
	GoTo directory option added to file manager to assist navigation.
	
    The database manager is cached between sessions (this can is on by default 
    and can be switched off with -Ddatabase_manager_cache_off). There is an 
    option under the File menu to clear this cache.
    
    A checkbox has been added to the window for adding ortholog/paralog links
    in the Gene Builder. If this is selected it adds links between existing 
    ortholog/paralogs and the new ortholog/paralog. By default this is off.
    
    An option has been added to the database manager to display polypeptide 
    domains in the feature display (as well as the protein map in the Gene
    Builder).
    
    An option has been added to the Run menu for doing a search of the Pfam
    database.
    
	Added an option to the Write menu for writing a combination of upstream 
	+ feature + downstream bases for selected features.

    Option added in the View->Filter Features menu to search for Duplicate
    Systematic Name Qualifier.

    LookSeq analysis panel can be displayed by setting the lookseq 
    value in the options file. An option under the Display menu then
    is used to shows the LookSeq read alignment panel in Artemis.

    Added options to set the minimum and maximum values of the plots.

    Transfer Annotation Tool (TAT) added to feature editor and Gene
    Builder.

	New graph popup menu option to show the values and average for
	a selected range.
	
	Added product_cv database option to define if the product is
	stored as a controlled vocabulary or as a feature property (featureprop).

Version 11

    Use black_belt_mode to suppress warnings when opening Artemis.
    
    Script (writedb_entry) added to make it easier to write out 
    multiple entries as EMBL / GFF files from chado. This uses Artemis 
    read-write libraries and does not require each sequence to be 
    launched from the database. For command line help run: 
    etc/writedb_entry -help

	On MacOSX - enable dropping files on Artemis application to open sequences
	with file extensions: gff, embl, EMBL, genbank, gbk, fasta, seq, art and 
	dna (defined in Artemis.app/Contents/Info.plist).

	Add NCBI search link to run menu. This transfers the sequence automatically
	to the NCBI web page.
	
	Add support for writing Sequin table format.
	
	Added print to PostScript option in Artemis and ACT.

    Optimisation of reading in user graphs.

	New Edit->Selected Feature(s)->Convert Keys option to convert
	keys of selected features.

	New Edit menu option for finding and replacing qualifier text. This
	has an option for boolean searches (e.g. and, or, &, |) of qualifier
	text. This includes an option to search for duplicate qualifiers.

	Implemented a commit manager for the database mode. This
	highlights transactions that produce an error.
	
	Make the chado transaction log messages more human readable.

    Add ability to write file formats from Artemis in database mode.
    With option to collapse the gene hierarchy (gene, transcript, exon) 
    into a CDS feature.

	Graphs are now added to a split pane. So that their size can be 
	defined by dragging the divider at the bottom of the graphs.

	Provide option to log transform user data plot.
	
	Improved error reporting for contig reordering.

   	Fix for creating intergenic features for overlapping CDS's. Also
   	add note based on which one of the 4 cases with respect to the 
   	flanking CDS it belongs to, i.e.:
 	IGR-F (forward):  cds> IGR cds>
	IGR-R (reverse): <cds IGR <cds
	IGR-B (both): <cds IGR cds>
	IGR-X: cds> IGR <cds
	
	Add option to preferences for defining contig ordering features.

	Added option to "Create features from graph peaks". For a graph 
	this creates features in regions above a given cut-off and above 
	a given feature size.
	
	Add -Dread_only option for read only databases.
	
	Add option to lazy load feature data from database.
	
Version 10

	Add redo function to 'Edit' menu. Also enable/disable undo and
	redo menu items when available/not available.
	
	Add option to replace selected bases in 'Edit' menu.
	
	Option to create features in intergenic regions added.

	Feature editor now marks hyperlinks to SWALL, EMBL, UniProt, 
	PMID, PubMed, InterPro, OrthoMCLDB, Pfam that are opened in 
	the browser. Now configured in the options file.
	
	Added "Convert Qualifier of Selected..." option to the Edit menu.
	This allows the user to change the names of qualifiers for all
	selected features.
	
	Now using new release of j2ssh (0.2.9). This requires Java1.5+.

	Implemented option for ORF creation to take into account boundaries
	of multiple fasta sequences, so that they do not cross them.

	Implemented the ability to run and store fasta and blast search 
	results for multiple databases... E.g. fasta searches on uniprot 
	and on user's own database, stored in multiple fasta_file qualifiers.

 	"Set Score Cutoffs" in Artemis popup menu uses existing /scores as
 	the initial min and max values (rather than just 0 and 100).

	Added cache to store the entries retrieved for the object editor.
	
	Implement log4j logging to be displayed in log viewer. Using colour
	coding depending on level of logging.
	
	Added -Doffset so that Artemis can be opened at a given base.

Version 9

	Feature selector can be used to look for features with introns 
	that do no contain the GT/GC start or AG end.

	Contig tool now checks for contigs that contain features that
	span the boundaries of the contigs. These features have to be 
	removed or restricted to the contig boundary before it can 
	carry out contig reordering.

	Fix for converting files from other file formats to
	genbank format.

	Fix rounding problem for long sequences when writting 
	out all bases in FASTA or raw format.

	Cache the start codons (as per stop codon caching),
	to speed their display.

	Combine the extend to next exon and the fix stop codons
  	into one option.

	The feature types that appear on the frame lines can be defined
	by the user via an option ("Frame Line Features...") in the 
	feature display popup menu.
	
	Added to File -> Preferences a user defined selection for display names
	and systematic names. Also extended popup menu option in feature lists 
	to allow the user to be able to select multiple qualifiers to display.

	For entries opened from the remote side of an SSH connection will search
	for results on the remote file system if they are not found locally. They
	are transferred via SSH and then stored locally.

	For Mac users, the option to send search results to the browser will
	display the results in the default browser.

	When automatically generating gene names (under the Edit menu), the user
	can specify the number of zeros to pad the numbering with. e.g if 5 digits are
	selected the format will look like : 00001, 00002 etc.