First, I want to say that I'm really impressed with the speed of loc! Now on to the issue:

I noticed that when I run "loc ." in the project root that the directory's git directory is included in the results. I propose either disabling git directories or disabling hidden directories from the results. Thoughts?

Tried installing twice. Build fails with E0432 and E0061 errors [log gist]. There aren't any issues in mmap's repo that point to a similar problem, but the project's been dormant for 2 years.

Gherkin is a language commonly used for writing BDD style tests, with parsers in a dozen+ different languages. It is part of the Cucumber project.

https://github.com/cucumber/cucumber/wiki/Gherkin

From: https://github.com/vmchale/polyglot

polyglot: 143.2 ms loc: 171.8 ms tokei: 304.6 ms

It might be interesting to look at which optimizations they are using.

Invalid values of --sort will now be caught and reported. Added a
dependency on the edit_distance crate to catch typos and offer
suggestions. These are basically usability improvements.

Sometimes it is more convenient to preprocess a list of files to remove complicated patterns that are annoying or impossible to specify via the --include --exclude regexes as well as use different sources for which files are interesting (eg. running a Powershell script to retrieve all of the files in a VS solution).

This PR adds support for letting you pass a file path via -l, --list-file which should just contain a path per line that will be used instead of the normal target argument to fill out the list of files to count.

When running loc as loc . --files on the phpSimpleSAML project I get the following output

--------------------------------------------------------------------------------
 Language             Files        Lines        Blank      Comment         Code
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
 Bourne Shell             2           86           19            6           61
--------------------------------------------------------------------------------
|./bin/build-release.sh              62           15            3           44
|./bin/check-syntax.sh               24            4            3           17
--------------------------------------------------------------------------------
 PHP                    473        65104        10377        18319        36408
--------------------------------------------------------------------------------
|templates/authsources.php          402           58          461   4294967179
|es/saml/lib/IdP/SAML2.php         1149          180          188          781
|L/Metadata/SAMLParser.php         1498          267          493          738
|tlibinc/base_facebook.php         1437          140          561          736
|pleSAML/Configuration.php         1369          205          528          636
...

There seems to be something going wrong with 4294967179

This is on: MacOS El Capitan loc version 0.3.4 rustc 1.12.1 (d4f39402a 2016-10-19)

Example of it running:

loc (lang/docker) # cargo run -- ../test
    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
     Running `target/debug/loc ../test`
--------------------------------------------------------------------------------
 Language             Files        Lines        Blank      Comment         Code
--------------------------------------------------------------------------------
 Docker                   1            4            1            1            2
--------------------------------------------------------------------------------
 Total                    1            4            1            1            2
--------------------------------------------------------------------------------

Contents of file:

loc (lang/docker) # cat ../test/Dockerfile
# This is a testing comment

FROM ubuntu
RUN echo "Hello world!"

Closes #95

Cool tool! Just found it because sloccount refuses to work any more. This is far faster!

With the .rs file example here, tokei correctly reports 1 line of code, whereas loc incorrectly reports 14 lines of code.

Haskell also has nestable block comments:

{-

{-
Nested comment
-}

-- Note: still commented
fibs :: [Int]
fibs = 1 : 1 : zipWith (+) fibs (tail fibs)

-}

main :: IO ()
main = print [1..]

tokei correctly reports 2 lines of code, whereas loc incorrectly reports 5 lines.

This may be by design, but:

Write the following to a file named e.g. foo.py:

#!/bin/python

hello = "Count me!"

Run loc foo.py. You'll get the expected result, 1 line of Python. Now run mv foo.py foo and run loc foo. The result shows 0 lines of anything.

This may be out of scope for loc (detecting filetypes without an extension can be hard), but it does mess up counts for codebases with scripts named without extensions, e.g. run_task, etc.

What would be required for loc to return counts for files without extensions?

At the moment the Cargo.lock of the current release does not contain hashes - this is currently breaking our package in nixpkgs, where we need them for offline builds.

Bumps regex from 1.5.4 to 1.5.5.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

https://github.com/cgag/loc/blob/1e0c7f434ddfd51439e1d4eb126f31b7a04229d9/src/lib.rs#L668

According to my tests, the vector multis in this line of code has only one element in most cases. Calling the iter() function on a vector with only one element results in having a lot of redundant load operations. If we create a fast path for a vector with only one element, we can eliminate a lot of computation and memory operations, which in turn improves the performance of the program. Execution time decreased from 7.05s to 5.74s with my test input, which is a 22.8% speedup.

Here's the modified code for this count() function. (src/lib.rs: 597-714)

` pub fn count(filepath: &str) -> Count { let lang = lang_from_ext(filepath); let (singles, multis) = counter_config_for_lang(lang);

let mfile = File::open(filepath);
let mut file = match mfile {
    Ok(file) => file,
    Err(_) => {
        return Count::default();
    }
};
// TODO(cgag): set the size of this vec to size of the file + a byte? a reddit comment
// somewhere says fs::read will do this ofr you.
let mut bytes = vec![];
file.read_to_end(&mut bytes).expect("nani?!");

let mut c = Count::default();
let mut multi_stack: Vec<(&str, &str)> = vec![];

//***YYZ note: The fast path for vecotr with only one element
if multis.len() == 1 {
    let (start, end) = &multis[0];
    let start_len = start.len();
    let end_len   = end.len();
    'line: for byte_line in ByteLines(&bytes).lines() {
        let line = match std::str::from_utf8(byte_line) {
            Ok(s) => s,
            Err(_) => return Count::default(),
        };
        c.lines += 1;

        let line = line.trim_start();
        if line.is_empty() {
            c.blank += 1;
            continue;
        };

        if multi_stack.is_empty() {
            for single_start in singles.iter() {
                if line.starts_with(single_start) {
                    if multis.iter().any(|(m_start, _)| line.starts_with(m_start)) {
                        break;
                    }

                    c.comment += 1;
                    continue 'line;
                }
            }

            if multis.is_empty() {
                c.code += 1;
                continue 'line;
            }
        }

        if multi_stack.is_empty() && !multis.iter().any(|(start, end)| line.contains(start) || line.contains(end)) {
            c.code += 1;
            continue 'line;
        }

        let mut pos = 0;
        let mut found_code = 0;
        let line_len = line.len();
        let contains_utf8 = (0..line_len).any(|i| !line.is_char_boundary(i));

        'outer: while pos < line_len {
                if contains_utf8 {
                    for i in pos..pos + min(max(start_len, end_len) + 1, line_len - pos) {
                        if !line.is_char_boundary(i) {
                            pos += 1;
                            continue 'outer;
                        }
                    }
                }

                if pos + start_len <= line_len && &line[pos..pos + start_len] == *start {
                    pos += start_len;
                    multi_stack.push(multis[0]);
                    continue;
                }

                if !multi_stack.is_empty() {
                    let (_, mut end) = multi_stack.last().expect("stack last");
                    if pos+end.len() <= line_len && &line[pos..pos+end.len()] == end {
                        let _ = multi_stack.pop();
                        pos += end.len();
                    }
                } else if multi_stack.is_empty() && pos < line_len && !&line[pos..pos + 1].chars().next().expect("whitespace check").is_whitespace() {
                    found_code += 1;
                }
            // }
            pos += 1;
        }

        // TODO(cgag): can this ever be greater or was that just defensive coding
        if found_code >= multis.len() {
            c.code += 1;
        } else {
            c.comment += 1;
        }
    }
// ***YYZ note: Will run the original code if it has more than one element
}else{
    'line: for byte_line in ByteLines(&bytes).lines() {
        let line = match std::str::from_utf8(byte_line) {
            Ok(s) => s,
            // TODO(cgag): should we report when this happens?
            Err(_) => return Count::default(),
        };
        c.lines += 1;

        let line = line.trim_start();
        // should blanks within a comment count as blank or comment? This counts them as blank.
        if line.is_empty() {
            c.blank += 1;
            continue;
        };

        // if we match a single line comment, count it and go onto next line
        // TODO(cgag): is the multiline comment start symbol ever the shorter one?
        // if multi_stack.is_empty, then we're not currently in a multiline comment
        if multi_stack.is_empty() {
            for single_start in singles.iter() {
                if line.starts_with(single_start) {
                    // if this single_start is a prefix of a multi_start,
                    // make sure that the line doesn't actually start with the multi_start
                    // TODO(cgag): donm't do this check here
                    // TODO(cgag): this assumption that the multi-line comment is always the longer one
                    //             may well be a terrible one
                    if multis.iter().any(|(m_start, _)| line.starts_with(m_start)) {
                        break;
                    }

                    c.comment += 1;
                    continue 'line;
                }
            }

            if multis.is_empty() {
                c.code += 1;
                continue 'line;
            }
        }

        if multi_stack.is_empty() && !multis.iter().any(|(start, end)| line.contains(start) || line.contains(end)) {
            c.code += 1;
            continue 'line;
        }

        let mut pos = 0;
        let mut found_code = 0;
        let line_len = line.len();
        let contains_utf8 = (0..line_len).any(|i| !line.is_char_boundary(i));

        'outer: while pos < line_len {
            for multi in multis.iter() {
                let (start, end) = multi;
                let start_len = start.len();
                let end_len   = end.len();

                // TODO(cgag): this is almost ceratinly giving us incorrect results.  Say the
                // first multi is the longest.  If we advance position because the final byte
                // position of that multi hits unicode, we might have skipped over a perfectly
                // valid comment start that was unaffected by the unicode.
                if contains_utf8 {
                    for i in pos..pos + min(max(start_len, end_len) + 1, line_len - pos) {
                        if !line.is_char_boundary(i) {
                            pos += 1;
                            continue 'outer;
                        }
                    }
                }

                if pos + start_len <= line_len && &line[pos..pos + start_len] == *start {
                    pos += start_len;
                    multi_stack.push(*multi);
                    continue;
                }

                if !multi_stack.is_empty() {
                    let (_, mut end) = multi_stack.last().expect("stack last");
                    if pos+end.len() <= line_len && &line[pos..pos+end.len()] == end {
                        let _ = multi_stack.pop();
                        pos += end.len();
                    }
                } else if multi_stack.is_empty() && pos < line_len && !&line[pos..pos + 1].chars().next().expect("whitespace check").is_whitespace() {
                    found_code += 1;
                }
            }
            pos += 1;
        }

        // TODO(cgag): can this ever be greater or was that just defensive coding
        if found_code >= multis.len() {
            c.code += 1;
        } else {
            c.comment += 1;
        }
    }
}

c

} `

When there are special characters in xml, vue, html, etc., loc will panic, causing the process to exit and statistics to fail. The patch ignores only the error file.

Is there a way to view the differences between files/directories in loc like there is in Cloc? In Cloc, we can compare two directories/files and see how many codes have been added, deleted, or modified. Is that option available in loc? If so, could you please show me how to do it?