This version is still in development and is not considered stable yet. For the latest stable version, please use dengjen-tashkeel 3.x!

Per-binding usage

Usage covers the Rust and Python quick starts shared by every binding. This page goes one level deeper per target — lifecycle, error handling, and a minimal working example drawn from each binding’s own source and tests.

Rust

use dengjen_tashkeel::{create_inference_engine, do_tashkeel, DengjenTashkeelError};

let engine = create_inference_engine(None)?;
match do_tashkeel(&engine, "بسم الله الرحمن الرحيم", None, false) {
    Ok(diacritized) => println!("{diacritized}"),
    Err(DengjenTashkeelError::InputTooLong(max_len)) => {
        eprintln!("input exceeds {max_len} characters")
    }
    Err(other) => eprintln!("diacritization failed: {other}"),
}

DengjenTashkeelError (crates/core/src/lib.rs) is a 3-variant enum: InputTooLong(usize), InferenceError(String), and ModelLoadError(std::io::Error). create_inference_engine takes an Option<PathBuf> — None loads the model bundled at build time, Some(path) loads an external ONNX model from that path.

do_tashkeel(engine, text, taskeen_threshold, preprocessed) keeps its positional shape. do_tashkeel_with(engine, text, &options) takes a TashkeelOptions instead, built with TashkeelOptions::new().with_taskeen_threshold(0.95).with_preprocessed(true); the struct is #[non_exhaustive], so later settings do not break callers.

C ABI

The C ABI (crates/capi, header crates/capi/dengjen_tashkeel.h) exposes 4 functions: dengjen_tashkeel_init, dengjenTashkeelTashkeel, dengjenTashkeelTashkeelWithOptions, and dengjen_tashkeel_free_string. dengjenTashkeelTashkeelWithOptions takes a pointer to a DengjenTashkeelOptions struct (struct_size set to sizeof(DengjenTashkeelOptions), preprocessed, has_taskeen_threshold, taskeen_threshold) in place of positional settings; a null pointer means the defaults. The library reads only the fields struct_size covers and defaults the rest, so a caller built against an older header keeps working when later releases append fields; a struct_size below 4 is DENGJEN_TASHKEEL_INVALID_ARGUMENT. Calling dengjenTashkeelTashkeel or dengjenTashkeelTashkeelWithOptions without ever calling dengjen_tashkeel_init first also works — either lazily initializes the engine on first use with the bundled default model, after which dengjen_tashkeel_init reports ALREADY_INITIALIZED.

Errors are reported through an ExternError out-parameter ({ code: i32, message: *mut c_char }) rather than a return value; code == 0 means success. The codes, from the header:

Code Meaning

Code	Meaning
`1` (`DENGJEN_TASHKEEL_INPUT_TOO_LONG`)	A sentence exceeds `CHAR_LIMIT`
`2` (`DENGJEN_TASHKEEL_INFERENCE_ERROR`)	Model inference failed
`3` (`DENGJEN_TASHKEEL_MODEL_LOAD_ERROR`)	The ONNX model file is missing or unreadable (core `ModelLoadError`, an I/O failure reading the file). A file that is readable but not a valid ONNX model is reported as code `2`
`4` (`DENGJEN_TASHKEEL_ALREADY_INITIALIZED`)	`dengjen_tashkeel_init` was called after the engine was initialized, explicitly or lazily; the existing engine is kept
`5` (`DENGJEN_TASHKEEL_INVALID_ARGUMENT`)	`dengjenTashkeelTashkeel` or `dengjenTashkeelTashkeelWithOptions` received a null `text`, or the options' `struct_size` is below 4
`99` (`DENGJEN_TASHKEEL_UNKNOWN_ERROR`)	Declared in the header; the current Rust code never returns it
`-1` (`ErrorCode_PANIC`)	The Rust side panicked instead of returning an error

1 (DENGJEN_TASHKEEL_INPUT_TOO_LONG)

A sentence exceeds CHAR_LIMIT

2 (DENGJEN_TASHKEEL_INFERENCE_ERROR)

Model inference failed

3 (DENGJEN_TASHKEEL_MODEL_LOAD_ERROR)

The ONNX model file is missing or unreadable (core ModelLoadError, an I/O failure reading the file). A file that is readable but not a valid ONNX model is reported as code 2

4 (DENGJEN_TASHKEEL_ALREADY_INITIALIZED)

dengjen_tashkeel_init was called after the engine was initialized, explicitly or lazily; the existing engine is kept

5 (DENGJEN_TASHKEEL_INVALID_ARGUMENT)

dengjenTashkeelTashkeel or dengjenTashkeelTashkeelWithOptions received a null text, or the options' struct_size is below 4

99 (DENGJEN_TASHKEEL_UNKNOWN_ERROR)

Declared in the header; the current Rust code never returns it

-1 (ErrorCode_PANIC)

The Rust side panicked instead of returning an error

The engine is a process-global singleton, and inference runs on a bounded pool of ONNX sessions (SessionPool in crates/core). Concurrent dengjenTashkeelTashkeel calls from multiple threads are safe; a call blocks until a pooled session is free.

Both the returned diacritized string and a non-null ExternError.message are heap-allocated on the Rust side and must be released with dengjen_tashkeel_free_string — freeing them any other way, or freeing twice, is undefined behavior.

The repository’s own example, crates/capi/ffi_usage_example.py, drives the compiled shared library from Python via ctypes:

def tashkeel(text, taskeen_threshold=None, preprocessed=False):
    err = ExternError()
    threshold_ptr = (
        ctypes.pointer(ctypes.c_float(taskeen_threshold))
        if taskeen_threshold is not None
        else None
    )
    ptr = lib.dengjenTashkeelTashkeel(
        text.encode("utf-8"),
        threshold_ptr,
        preprocessed,
        ctypes.byref(err),
    )
    if err.code != 0:
        raise RuntimeError(err.take_message())
    try:
        return ctypes.cast(ptr, ctypes.c_char_p).value.decode("utf-8")
    finally:
        lib.dengjen_tashkeel_free_string(ptr)

See that file in full for the ExternError ctypes structure and the ctypes.cdll.LoadLibrary setup it depends on.

Python

The dengjen_tashkeel_py PyO3 extension exposes tashkeel(text, taskeen_threshold=None, preprocessed=None) → str and init(model_path=None). The engine loads lazily on the first tashkeel call, with the bundled model; call init first to load it up front or from a custom ONNX file. init raises RuntimeError when the engine is already loaded or the model cannot be loaded, the same rule as dengjen_tashkeel_init. tashkeel maps DengjenTashkeelError::InputTooLong to a ValueError and every other error variant to a RuntimeError (crates/python/src/lib.rs, to_py_err):

from dengjen_tashkeel_py import tashkeel

try:
    diacritized = tashkeel("بسم الله الرحمن الرحيم", taskeen_threshold=0.95)
except ValueError:
    ...  # a sentence longer than CHAR_LIMIT (12,000 characters)
except RuntimeError:
    ...  # inference failure, or the engine could not be loaded on first use

Java

Tashkeel.loadDefault() or Tashkeel.load(Path) initialize the native engine once (both throw TashkeelException on failure). The engine is process-global, so repeating a successful load with the same source returns the same instance; loading a different model, or switching between the bundled and a custom one, throws AlreadyInitialized. diacritize(text, Optional<Float> taskeenThreshold, boolean preprocessed) then diacritizes a string. TashkeelException.reason() is a sealed interface — InputTooLong, InferenceError, ModelLoadError, AlreadyInitialized, InvalidArgument, and Unknown, each a record carrying the underlying message (Unknown also carries the raw code). Codes 1 to 5 map to the first five in that order; any other code maps to Unknown(code, message) (Tashkeel.reasonFor). So callers can exhaustively switch over the failure without a default branch:

import io.github.zirekhq.dengjen.tashkeel.Tashkeel;
import io.github.zirekhq.dengjen.tashkeel.TashkeelException;
import java.util.Optional;

try (Tashkeel tashkeel = Tashkeel.loadDefault()) {
    String diacritized = tashkeel.diacritize("بسم الله الرحمن الرحيم", Optional.empty(), false);
} catch (TashkeelException e) {
    String detail = switch (e.reason()) {
        case TashkeelException.InputTooLong r -> "input too long: " + r.message();
        case TashkeelException.InferenceError r -> "inference failed: " + r.message();
        case TashkeelException.ModelLoadError r -> "model load failed: " + r.message();
        case TashkeelException.AlreadyInitialized r -> "already initialized: " + r.message();
        case TashkeelException.InvalidArgument r -> "invalid argument: " + r.message();
        case TashkeelException.Unknown r -> "unknown error " + r.code() + ": " + r.message();
    };
}

Tashkeel is AutoCloseable for forward compatibility, but close() is currently a no-op: the native library has no per-instance teardown, only a process-global singleton set up by load. Remember --enable-native-access=ALL-UNNAMED on the JVM invocation (see Installation) or the Foreign Function & Memory API calls emit a native-access warning.

CLI

Beyond the flags reference in Usage, --input-file/-f changes how input is batched: with -f, the CLI reads the whole file and diacritizes each line independently, writing one output line per input line. Without it, the CLI reads stdin one line at a time and diacritizes each line as its own call; in interactive mode (the default when neither -f nor -o is given) it keeps reading until EOF, and with only -o it reads all of stdin (crates/cli/src/main.rs, get_input_text, tashkeel_main).

The CLI does not truncate input: a line containing a single sentence longer than CHAR_LIMIT (12,000 characters) fails with the core InputTooLong error; insert sentence breaks before passing such lines in. Log verbosity is set with the TASHKEEL_LOG environment variable (an env_logger filter, default info).

printf 'بسم الله الرحمن الرحيم\nالحمد لله رب العالمين\n' > input.txt
dengjen-tashkeel --input-file input.txt --output-file output.txt --taskeen --prob 0.9

This diacritizes each of the 2 lines in input.txt on its own, using taskeen at a 0.9 confidence threshold, and writes the 2 results to output.txt.