Count your code, quickly.

Last update: Jan 9, 2023

Related tags

System tools windows macos linux rust cli statistics code badge tokei cloc

Overview

Tokei (時計)

Tokei is a program that displays statistics about your code. Tokei will show the number of files, total lines within those files and code, comments, and blanks grouped by language.

Translations

中文

Example

===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 BASH                    4           49           30           10            9
 JSON                    1         1332         1332            0            0
 Shell                   1           49           38            1           10
 TOML                    2           77           64            4            9
-------------------------------------------------------------------------------
 Markdown                5         1355            0         1074          281
 |- JSON                 1           41           41            0            0
 |- Rust                 2           53           42            6            5
 |- Shell                1           22           18            0            4
 (Total)                           1471          101         1080          290
-------------------------------------------------------------------------------
 Rust                   19         3416         2840          116          460
 |- Markdown            12          351            5          295           51
 (Total)                           3767         2845          411          511
===============================================================================
 Total                  32         6745         4410         1506          829
===============================================================================

Features

Tokei is very fast, and is able to count millions of lines of code in seconds. Check out the 12.0.0 release to see how Tokei's speed compares to others.
Tokei is accurate, Tokei correctly handles multi line comments, nested comments, and not counting comments that are in strings. Providing an accurate code statistics.
Tokei has huge range of languages, supporting over 150 languages, and their various extensions.
Tokei can output in multiple formats(CBOR, JSON, YAML) allowing Tokei's output to be easily stored, and reused. These can also be reused in tokei combining a previous run's statistics with another set.
Tokei is available on Mac, Linux, and Windows. See installation instructions for how to get Tokei on your platform.
Tokei is also a library allowing you to easily integrate it with other projects.

Installation

Package Managers

Linux

# Alpine Linux (since 3.13)
apk add tokei
# Arch Linux
pacman -S tokei
# Cargo
cargo install tokei
# Conda
conda install -c conda-forge tokei
# Fedora
sudo dnf install tokei
# FreeBSD
pkg install tokei
# NetBSD
pkgin install tokei
# Nix/NixOS
nix-env -i tokei
# OpenSUSE
sudo zypper install tokei

macOS

# Homebrew
brew install tokei
# MacPorts
sudo port selfupdate
sudo port install tokei

Windows

scoop install tokei

Manual

Downloading

You can download prebuilt binaries in the releases section.

Building

You can also build and install from source (requires the latest stable Rust compiler.)

cargo install --git https://github.com/XAMPPRocky/tokei.git

Configuration

Tokei has a configuration file that allows you to change default behaviour. The file can be named tokei.toml or .tokeirc. Currently tokei looks for this file in three different places. The current directory,your home directory, and your configuration directory.

How to use Tokei

Basic usage

This is the basic way to use tokei. Which will report on the code in ./foo and all subfolders.

$ tokei ./foo

Multiple folders

To have tokei report on multiple folders in the same call simply add a comma, or a space followed by another path.

$ tokei ./foo ./bar ./baz

$ tokei ./foo, ./bar, ./baz

Excluding folders

Tokei will respect all .gitignore and .ignore files, and you can use the --exclude option to exclude any additional files. The --exclude flag has the same semantics as .gitignore.

$ tokei ./foo --exclude *.rs

Paths to exclude can also be listed in a .tokeignore file, using the same syntax as .gitignore files.

Sorting output

By default tokei sorts alphabetically by language name, however using --sort tokei can also sort by any of the columns.

blanks, code, comments, lines

$ tokei ./foo --sort code

Outputting file statistics

By default tokei only outputs the total of the languages, and using --files flag tokei can also output individual file statistics.

$ tokei ./foo --files

Outputting into different formats

Tokei normally outputs into a nice human readable format designed for terminals. There is also using the --output option various other formats that are more useful for bringing the data into another program.

Note: This version of tokei was compiled without any serialization formats, to enable serialization, reinstall tokei with the features flag.

  ALL:
  cargo install tokei --features all

  JSON:
  cargo install tokei --features json

  CBOR:
  cargo install tokei --features cbor

  YAML:
  cargo install tokei --features yaml

Currently supported formats

JSON --output json
YAML --output yaml
CBOR --output cbor

$ tokei ./foo --output json

Reading in stored formats

Tokei can also take in the outputted formats added in the previous results to it's current run. Tokei can take either a path to a file, the format passed in as a value to the option, or from stdin.

$ tokei ./foo --input ./stats.json

Options

USAGE:
    tokei [FLAGS] [OPTIONS] [--] [input]...

FLAGS:
    -f, --files               Will print out statistics on individual files.
    -h, --help                Prints help information
        --hidden              Count hidden files.
    -l, --languages           Prints out supported languages and their extensions.
        --no-ignore           Don't respect ignore files (.gitignore, .ignore, etc.). This implies --no-ignore-parent,
                              --no-ignore-dot, and --no-ignore-vcs.
        --no-ignore-dot       Don't respect .ignore and .tokeignore files, including those in parent directories.
        --no-ignore-parent    Don't respect ignore files (.gitignore, .ignore, etc.) in parent directories.
        --no-ignore-vcs       Don't respect VCS ignore files (.gitignore, .hgignore, etc.), including those in parent
                              directories.
    -V, --version             Prints version information
    -v, --verbose             Set log output level:
                                          1: to show unknown file extensions,
                                          2: reserved for future debugging,
                                          3: enable file level trace. Not recommended on multiple files

OPTIONS:
    -c, --columns <columns>       Sets a strict column width of the output, only available for terminal output.
    -e, --exclude <exclude>...    Ignore all files & directories matching the pattern.
    -i, --input <file_input>      Gives statistics from a previous tokei run. Can be given a file path, or "stdin" to
                                  read from stdin.
    -o, --output <output>         Outputs Tokei in a specific format. Compile with additional features for more format
                                  support. [possible values: cbor, json, yaml]
    -s, --sort <sort>             Sort languages based on column [possible values: files, lines, blanks, code, comments]
    -t, --type <types>            Filters output by language type, seperated by a comma. i.e. -t=Rust,Markdown

ARGS:
    <input>...    The path(s) to the file or directory to be counted.

Badges

Tokei has support for badges. For example .

[![](https://tokei.rs/b1/github/XAMPPRocky/tokei)](https://github.com/XAMPPRocky/tokei).

Tokei's URL scheme is as follows.

https://tokei.rs/b1/{host: values: github|gitlab}/{Repo Owner eg: XAMPPRocky}/{Repo name eg: tokei}

By default the badge will show the repo's LoC(Lines of Code), you can also specify for it to show a different category, by using the ?category= query string. It can be either code, blanks, files, lines, comments, Example show total lines:

[![](https://tokei.rs/b1/github/XAMPPRocky/tokei?category=lines)](https://github.com/XAMPPRocky/tokei).

The server code hosted on tokei.rs is in XAMPPRocky/tokei_rs

Plugins

Thanks to contributors tokei is now available as a plugin for some text editors.

Vim by vmchale

Supported Languages

If there is a language that you would to add to tokei feel free to make a pull request. Languages are defined in languages.json, and you can read how to add and test your language in our CONTRIBUTING.md.

Abap
ActionScript
Ada
Agda
Alex
Alloy
Asn1
Asp
AspNet
Assembly
AssemblyGAS
Autoconf
AutoHotKey
Automake
Bash
Batch
BrightScript
C
Cabal
Cassius
Ceylon
CHeader
Clojure
ClojureC
ClojureScript
CMake
Cobol
CoffeeScript
Cogent
ColdFusion
ColdFusionScript
Coq
Cpp
CppHeader
Crystal
CSharp
CShell
Css
D
DAML
Dart
DeviceTree
Dhall
Dockerfile
DotNetResource
DreamMaker
Dust
Edn
Elisp
Elixir
Elm
Elvish
EmacsDevEnv
Emojicode
Erlang
FEN
Fish
FlatBuffers
Forth
FortranLegacy
FortranModern
FreeMarker
FSharp
Fstar
GDB
GdScript
Gherkin
Gleam
Glsl
Go
Graphql
Groovy
Gwion
Hamlet
Handlebars
Happy
Haskell
Haxe
Hcl
Hex
Hlsl
HolyC
Html
Idris
Ini
IntelHex
Isabelle
Jai
Java
JavaScript
Json
Jsx
Julia
Julius
KakouneScript
Kotlin
Lean
Less
LinkerScript
Liquid
Lisp
LLVM
Logtalk
Lua
Lucius
Madlang
Makefile
Markdown
Meson
Mint
ModuleDef
MoonScript
MsBuild
Mustache
Nim
Nix
NotQuitePerl
ObjectiveC
ObjectiveCpp
OCaml
Odin
Org
Oz
Pascal
Perl
Perl6
Pest
Php
Polly
Pony
PostCss
PowerShell
Processing
Prolog
Protobuf
PSL
PureScript
Python
Qcl
Qml
R
Racket
Rakefile
Razor
Renpy
ReStructuredText
RON
RPMSpecfile
Ruby
RubyHtml
Rust
Sass
Scala
Scheme
Scons
Sh
Sml
Solidity
SpecmanE
Spice
Sql
SRecode
Stratego
Svelte
Svg
Swift
Swig
SystemVerilog
Tcl
Tex
Text
Thrift
Toml
Tsx
Twig
TypeScript
UnrealDeveloperMarkdown
UnrealPlugin
UnrealProject
UnrealScript
UnrealShader
UnrealShaderHeader
UrWeb
UrWebProject
Vala
VB6
VBScript
Velocity
Verilog
VerilogArgsFile
Vhdl
VimScript
VisualBasic
VisualStudioProject
VisualStudioSolution
Vue
WebAssembly
Wolfram
Xaml
XcodeConfig
Xml
XSL
Xtend
Yaml
Zig
Zsh

Common issues

Tokei says I have a lot of D code, but I know there is no D code!

This is likely due to gcc generating .d files. Until the D people decide on a different file extension, you can always exclude .d files using the -e --exclude flag like so

$ tokei . -e *.d

Canonical Source

The canonical source of this repo is hosted on GitHub. If you have a GitHub account, please make your issues, and pull requests there.

Copyright and License

See the graph for a full list of contributors.

Tokei is distributed under the terms of both the MIT license and the Apache License (Version 2.0).

See LICENCE-APACHE, LICENCE-MIT for more information.

Comments

Show Inaccuracy Warning on permission error

This PR roughly implements issue #165. I am sure there is room for some improvements on my code changes so feel free to comment on everything I changed. One thing I dislike that I did is this weird if let bundle I created to check for a permission error at one place: https://github.com/Veykril/tokei/blob/0c7e18d1b0b45832c40c3a991be4bd57111ff5e5/src/utils/fs.rs#L52-L61 I couldnt come up with a better way since the nested errors are boxed and box patterns arent stable yet.

Example Output:

C:\Workspace\Rust\tokei (error_output -> origin)                                                                       
λ cargo run -- C:\Workspace\Rust\test                                                                                  
    Finished dev [unoptimized + debuginfo] target(s) in 0.18s                                                          
     Running `target\debug\tokei.exe C:\Workspace\Rust\test`                                                           
ERROR 2018-10-01T16:28:56Z: tokei::utils::fs: Permission denied for reading C:\Workspace\Rust\test\Cargo.toml          
Note: results can be inaccurate for languages marked with '(!)'                                                        
-----------------------------------------------------------------------------------------------------------------------
 Language                                                    Files        Lines         Code     Comments       Blanks 
-----------------------------------------------------------------------------------------------------------------------
 Markdown                                                        4         1380         1380            0            0 
 Rust                                                            1          123           99            1           23 
 Shell                                                           1           40           29            1           10 
 TOML (!)                                                        1           84           69            0           15 
 YAML                                                            1           80           60            8           12 
-----------------------------------------------------------------------------------------------------------------------
 Total (!)                                                        8         1707         1637           10           60
-----------------------------------------------------------------------------------------------------------------------

opened by Veykril 15

Use box drawing characters?

Skimming through the new comparison between tools, I noticed how all of them use the visually noisy hyphen ----- lines. The box drawing characters could be used to have clean, uninterrupted lines. These improve aesthetics and improve readability. Of course, this requires unicode support. I wasn't sure if the choice of hyphens was deliberate to support legacy terminals.

Some comparisons:

-----------------------------------------------------------------------------------
 Language                Files        Lines         Code     Comments       Blanks
-----------------------------------------------------------------------------------
 Dockerfile                 14          251          219            6           26
 Markdown                    3          180          180            0            0
 Rust                      102        41073        28770         8630         3673
 Shell                       6          310          195           63           52
 TOML                        8          141          120            3           18
 XML                         1       134861       133755            0         1106
-----------------------------------------------------------------------------------
 Total                     134       176816       163239         8702         4875
-----------------------------------------------------------------------------------

───────────────────────────────────────────────────────────────────────────────────
 Language                Files        Lines         Code     Comments       Blanks
───────────────────────────────────────────────────────────────────────────────────
 Dockerfile                 14          251          219            6           26
 Markdown                    3          180          180            0            0
 Rust                      102        41073        28770         8630         3673
 Shell                       6          310          195           63           52
 TOML                        8          141          120            3           18
 XML                         1       134861       133755            0         1106
───────────────────────────────────────────────────────────────────────────────────
 Total                     134       176816       163239         8702         4875
───────────────────────────────────────────────────────────────────────────────────

Following LaTeX booktabs style:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  Language                Files        Lines         Code     Comments       Blanks
───────────────────────────────────────────────────────────────────────────────────
 Dockerfile                 14          251          219            6           26
 Markdown                    3          180          180            0            0
 Rust                      102        41073        28770         8630         3673
 Shell                       6          310          195           63           52
 TOML                        8          141          120            3           18
 XML                         1       134861       133755            0         1106
───────────────────────────────────────────────────────────────────────────────────
 Total                     134       176816       163239         8702         4875
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

enhancement help wanted Easy

opened by Emerentius 14

Number of files tallying doesn't work

It seems piping output around doesn't update the file total, plus, only the file types from the final command are being tallied.

~$  tokei usercorn-unstable/
-------------------------------------------------------------------------------
 Language            Files        Lines         Code     Comments       Blanks
-------------------------------------------------------------------------------
 Go                    149         9163         7886          298          979
 Makefile                1           26           18            0            8
 Markdown                3          254          254            0            0
 Python                  2          154          125            2           27
 YAML                    1           20           17            0            3
-------------------------------------------------------------------------------
 Total                 156         9617         8300          300         1017
-------------------------------------------------------------------------------


~$  tokei go/src/github.com/
-------------------------------------------------------------------------------
 Language            Files        Lines         Code     Comments       Blanks
-------------------------------------------------------------------------------
 BASH                    1            8            6            1            1
 C Header                6         3435         2939          400           96
 Go                    170        29590        25755         1492         2343
 Markdown               13         1090         1090            0            0
 Python                  2          130          101            9           20
 YAML                    6           69           65            0            4
-------------------------------------------------------------------------------
 Total                 198        34322        29956         1902         2464
-------------------------------------------------------------------------------

~$  tokei -o json go/src/github.com/ | tokei -i stdin usercorn-unstable/
-------------------------------------------------------------------------------
 Language            Files        Lines         Code     Comments       Blanks
-------------------------------------------------------------------------------
 Go                    149        38753        33641         1790         3322
 Makefile                1           26           18            0            8
 Markdown                3         1344         1344            0            0
 Python                  2          284          226           11           47
 YAML                    1           89           82            0            7
-------------------------------------------------------------------------------
 Total                 163        43939        38256         2202         3481
-------------------------------------------------------------------------------

opened by MagaTailor 13

12.0.4 and master segfault on Windows, when built with optimizations.

I am installing and testing master and 12.0.4 using the current stable compiler 1.45.2. All tests are passing. But running tokei results in STATUS_ACCESS_VIOLATION (segfault). However the debug build seems to run fine (So it could be caused through optimizations?). When debugging it (release mode + debug info), it seems that it is crashing inside of the ignore crate, in the walk.rs file, in the visit method, on line 1293, where all the thread handles are joined. [link] This seems to be called by your utils/fs.rs, in the get_all_files method, on line 57, where you build and run the parallel file walkers. [link]

This was an issue in the past: #543
bug

opened by clemenswasser 12
Introduce `Bytes` to support processing of non-UTF-8 files.
I was trying to use tokei to analyze older versions of the Linux kernel and noticed that it had issues with files containing comments which were encoded in iso-8859-1.

Because DecodeReaderBytes wasn't actually configured to use any encoding it fell back to using UTF-8, unless a BOM was present. BOMs are exceedingly rare when dealing with code files in my experience, but it's still something that we should re-incorporate here before merging to maintain the old behavior. ~~My suggestion would be to perform an explicit check of the two first bytes in the stream~~ (see TODO).

Unfortunately file encodings tend to be determined by build systems rather than the files themselves. But we can do other things like look for vim comments (e.g. # vim: set fileencoding=).

While reading through the code, I noticed that the analysis performed by tokei didn't use a ton of string APIs, so I built the Bytes abstraction to handle all the cases that were needed to operate directly on top of a &[u8] slice.

Any char inspection that we do now tries to decode a line using a method called utf8_chars_lossy(), which replaces any illegal utf-8 sequences with a replacement character U+FFFD.

TODO

[x] Check BOM.

[x] Test for binary files

this was done before because DecodeReaderBytes always attempted to decode at least as UTF-8.
opened by udoprog 10
Unrecognized/unsupported filetypes

If a file don't have either an extension or a shebang, tokei should count it as plain text. If a file has an extension or a shebang, but it is not known by tokei, it should count it as Unsupported, and use the same rules as used with plain text (every line is code).

opened by notramo 10

Made a series of modifications to the languages comments:

Clojure - Removed #
Forth   - ( Comment) style comments need a space after the opening paren
Haskell - Added multiline comment style {- Block comment -}
Jai     - Has nested block comments
Julia   - Has nested block comments
Kotlin  - Has nested block comments
Pascal  - Pascal should be multiline from { or (* to } or *)
Perl    - Perl5 and earlier for multiline comments need =pod to =cut.
            Perl6 will be different but as it is not finished I'm not going
            to add it.
Python  - Python can use """ or ''' for comments
Swift   - Has nested block comments

The is_fortran code was editted by rustfmt.

opened by mwilli20 10

Update existing languages with their String litreals.
Now with the inclusion handling comments while in quotes, and out of them. All existing languages need to be updated with their string type as currently the default is (", ") which obviously doesn't apply to all languages. All that is needed to go to src/lib/language/languages.rs and use the set_quotes() method.

[ ] ActionScript

[ ] Ada

[x] Assembly

[ ] Autoconf

[x] BASH

[x] Batch

[x] C

[x] C Header

[x] Clojure

[x] CoffeeScript

[x] ColdFusion

[x] ColdFusion CFScript

[ ] Coq

[x] C++

[x] C++ Header

[ ] C#

[ ] C Shell

[x] CSS

[ ] D

[x] Dart

[ ] Device Tree

[ ] Erlang

[ ] Forth

[ ] FORTRAN Legacy

[ ] FORTRAN Modern

[ ] Go

[ ] Handlebars

[x] Haskell

[x] HTML

[x] Idris

[ ] Isabelle

[x] JAI

[x] Java

[x] JavaScript

[x] Julia

[x] JSON

[ ] JSX

[x] Kotlin

[ ] LESS

[ ] LD Script

[ ] LISP

[ ] Lua

[ ] Makefile

[x] Markdown

[ ] Mustache

[ ] Nim

[ ] Objective C

[ ] Objective C++

[ ] OCaml

[ ] Oz

[ ] Pascal

[ ] Perl

[ ] Polly

[x] PHP

[ ] Protocol Buffers

[ ] Prolog

[ ] Python

[ ] QCL

[ ] R

[ ] Ruby

[ ] Ruby HTML

[x] Rust

[ ] Sass

[ ] Scala

[ ] Standard ML

[ ] SQL

[ ] Swift

[ ] TeX

[x] Plain Text

[ ] TOML

[ ] TypeScript

[ ] Vim Script

[ ] Unreal Script

[ ] Wolfram

[ ] XML

[ ] YAML

[x] Zsh

help wanted
opened by XAMPPRocky 10
Add Terra support and Lua testing

This pull adds support for Terra, a strict-typed systems language that is metaprogrammable in Lua, as well as testing support for both Terra and Lua. I also took the liberty of adding more of the blockquote and block-comment variants Lua supports (you can have arbitrarily many =s between the square brackets to produce different levels of nesting), and moved the blockquotes to verbatim, as they can include literal "s.

opened by seylerius 9
Optionally format numbers with thousands separator

Closes #256.

Add a new option, -n/--num-format to specify whether the output should separated by commas, dots, or nothing (plain).

This adds a new import, num-format, to perform the formatting. This adds a total of three new dependencies: arrayvec version 0.4, ndodrop, and num-format.

If we like the way this looks, I can add a configuration option. I would assume that most users would prefer to put their preferred value in the configuration option so they don't have to specify it every time.

opened by HarrisonMc555 9
Ability to use with non-filesystem files
I'd like to analyze files that are in tarballs and bare git checkouts, on the fly, without copying these files to disk.

Could you expose an interface that would make it possible? e.g.

let mut tokei = Tokei::new(); for imaginary_path in fake_filesystem { tokei.add_file(imaginary_path, bytes_of_file_body); // e.g. "file.rs", &[u8] } let counts = tokei.totals();
opened by kornelski 9
Add support for jq
This adds the jq language

comments: https://github.com/stedolan/jq/wiki/jq-Language-Description#jq-program-structure-and-basic-syntax

strings: https://github.com/stedolan/jq/wiki/jq-Language-Description#data-types
opened by glennj 0
add `gradlew` for default `.tokeignore`
when working on gradle projects, gradlew scripts are vendored together. it would be great if tokei could default ignore following files.

gradlew gradlew.bat
opened by scarf005 0
Add Max support

Visual programming language for music and multimedia, for details see:

What is Max? | Cycling '74 https://cycling74.com/products/max

Max (software) - Wikipedia https://en.wikipedia.org/wiki/Max_(software)

Note: maxpat's are json code, which do not support comments, thus I skipped adding a test file

Thanks a lot for this great piece of software!

opened by mbyt 0
Make language entries colorable

I think having languages colored is a good way of quickly distinguishing them. So I went ahead and tried to implemented it.

I gave my best to implement it in a way compatible with the project structure.

I already added colors for JavaScript, HTML and Svelte files. If the code looks good I would also add colors for all the existing languages before merging, I just wanted to confirm the functionality and implementation with a maintainer. HTML and JavaScript make it possible to manually test if the code is working.

It works by adding a key in the languages.json: "color": {"r": number, "g": number, "b": number} and if none is defined it just does not apply any color.

This PR should also close #835.

opened by sbatial 4