Using fparser in FORD documentation and fortls language server

[gnikit]

Hi, I am the dev behind the fortls Fortran Language Server. There has been a few discussions in the Fortran community [1, 2] about utilising a single Fortran parser to fit our needs and I was wondering whether or not fparser could be used.

My main concerns with fparser are

• its lack of support for F2018 standard
• Relatively slow performance when compared to fortls's or lfortran's AST parsers

Do you have any plans on extending the supported standard to F2018?
And do you see a potential overlap between fortls and/or FORD and fparser

CC @ZedThree

[rupertford]

Hi @gnikit,

Thanks for your interest in fparser.

We have been adding F2018 support as and when it has been needed, so there is already some support and I don't see any blockers in adding full support.

We would obviously be pleased if fortls and/or FORD use fparser so if full F2018 support is something that you need then we could look to prioritise that.

As for performance, one of my colleagues (@sergisiso) has some ideas on how speed things up (beyond better tree pruning by tightening the rule checking and traversing in the best order), but the fparser approach of building a full parse tree through recursive rule checking is probably going to be slower than more lightweight approaches. I've not compared the speed of lfortran and fparser so don't know. Perhaps they benefit from being written in C++ cf Python? It would be interesting if you had some benchmark codes and timings for us to take a look at.

As for overlap, I can see that fparser could potentially be used in fortls (and FORD) but beyond that they are pretty separate (if that is what you meant).

We have a telco planned with @ZedThree to discuss fparser and FORD further. Perhaps we could arrange one to discuss fparser and fortls if you think that would be useful?

[gnikit]

Hi @rupertford,

Thanks for the insightful feedback, it is much appreciated. I am very happy to see that you are actively developing the project and are thinking about performance improvements. I have some performance timings that I will attach bellow.

The F2018 is not a major issue IMO as long as the parser does not break catastrophically. Basically, the unintuitive part of a Language Server is that it requires an error tolerant parser, since we cannot guarantee valid Fortran syntax (and hence ASTs) whilst typing Fortran code.

Perhaps we could arrange one to discuss fparser and fortls if you think that would be useful?

A chat would be great. Please contact me at gnikit@duck.com

Benchmark

This is a fairly simple F90 benchmark over a very large file. certik suggested it to me when we compared lfortran's parser with fortls's

Python code to generate benchmark source code

def gen_sub(no: int):
    sub = """subroutine g{0}(x)
    integer, intent(inout) :: x
    integer :: i
    x = 0
    do i = {0}, {1}
        x = x+i
    end do
end subroutine

""".format(
        no, no + 9
    )
    return sub


def write_bench3():
    with open("bench3.f90", "w") as f:
        f.write("program bench3\n")
        f.write("implicit none\n")
        f.write("integer :: c\n")
        f.write("c = 0\n")
        for i in range(1, 10001):
            f.write(f"call g{i}(c)\n")
        f.write("\nprint *, c\n")
        f.write("\ncontains\n\n")
        for i in range(1, 10001):
            f.write(gen_sub(i))
        f.write("end program")


if __name__ == "__main__":
    write_bench3()

time fparser2 --task=none bench3.f90

time gfortran -fdump-fortran-original -fsyntax-only bench3.f90

time fortls --debug_filepath bench3.f90 --debug_rootpath . --debug_parser

time lfortran --show-ast --no-color bench3.f90

I don't necessarily think this is a 100% fair comparison, but it should give a rough idea about the performance of each parser.
How fortls cheats its way out of its poor performance is by using multiple threads during parsing which helps a lot.

	lfortran	fortls	gfortran	fparser
time	0.094s	4.226s	5.543s	58.588s

[rupertford]

Thanks @gnikit. We are impressively slow :-). We have some ideas on how to improve so will work on those.