Development of a Stable Diffusion and ControlNet-Based Content Generator for the Digital Preservation of Javanese Script

I. Introduction

A. Motivation and research problem

We developed this study around a practical question: how can a generative system change the visual material of Javanese writing without losing the writing itself? Our experimental work brings together script preparation, spatial conditioning, and learned material styling [1]. At Data Sorcerers, we adapt this research for GAKSARA, a workflow for creating contemporary Javanese-script visual assets. The objective is to make the script usable in digital creative production while keeping its linguistic structure at the center of the process.

Javanese writing is more than a sequence of independent decorative shapes. The position of vowel signs, consonant modifiers, and pasangan forms affects how a sequence is read. Cropping a small mark, merging adjacent components, or inventing a stroke can alter an otherwise attractive result. Unicode encodes Javanese in the A980–A9DF block [2], but encoding alone does not determine a correct rendered layout. We therefore distinguish three responsibilities: selecting the intended characters, placing their glyphs correctly, and preserving their appearance during image synthesis.

A text-to-image prompt can describe gold, stone, or wood without explicitly specifying every contour of a Javanese word. TextDiffuser and related text-aware diffusion work show why layout and text rendering need explicit treatment [3]. Our response is an engineering decomposition: construct an inspectable script image first, then use that image to guide material synthesis. This reduces the amount of orthographic structure that the generative model must infer from a natural-language prompt.

B. Objectives and contributions

We organize the study around three questions. First, can normalized Latin input be converted into a centered, multi-line Javanese guide suitable for controlled generation? Second, how do training progress and the balance of spatial control and LoRA styling affect the visual result? Third, can these stages be integrated into a web workflow that allows a user to choose a style, inspect the output, and export an image?

Our contributions are a hybrid preprocessing and generation pipeline; a 70-pair synthetic collection covering five material styles; a qualitative study of checkpoints, six inference configurations, and extreme settings; and a working interface connecting text entry with generated output. The study supports digital cultural communication and creative production. It does not reconstruct historical inscriptions or authenticate manuscripts. We preserve that distinction when adapting the workflow for GAKSARA: a new image inspired by a cultural script remains a contemporary generated asset.

II. Background and technical foundations

A. Diffusion and latent image synthesis

Denoising diffusion models learn to reverse a sequence of noise-corruption steps [4]. Latent diffusion performs this process in an autoencoder representation rather than directly at full image resolution [5]. We use Stable Diffusion v1.5 as the image-generation backbone. Its role is to synthesize texture, illumination, and visual context from a text condition and spatial guidance. It is not used as the transliteration engine.

B. Spatial conditioning with ControlNet

ControlNet introduces trainable conditioning branches connected to a pretrained diffusion network through zero-initialized convolutions. The original method supports controls including edges and depth [6]. We combine an edge-conditioned branch with a depth-conditioned branch because a glyph perimeter and a foreground–background guide convey different spatial cues. Canny contours identify boundaries; the second guide identifies the filled region associated with the writing. Their influence is learned and adjustable, so the generated image still requires inspection.

C. Low-rank adaptation for material appearance

LoRA represents a weight update using low-rank matrices while retaining the pretrained weights [7]. In the conventional notation, a weight matrix is adapted as W = W₀ + BA, where A and B parameterize a lower-rank update. We use the adapter to specialize appearance toward the selected material styles. The inference-time LoRA strength controls the contribution of that learned adaptation; it is distinct from the rank used during training.

This separation is central to our design. The transliteration and layout stages establish a target structure, ControlNet encourages the diffusion output to follow that structure, and the adapter supplies material-specific appearance. Stronger style does not necessarily produce a better result: excessive texture may spread into the background or compete with small glyph details. Likewise, stronger structural conditioning can make the result rigid or visually flat. We investigate that balance empirically.

D. Script conversion and raster layout

Aksharamukha provides script conversion and script-specific orthographic processing [8]. We pair it with a Python normalization layer and use Pillow for text measurement and raster drawing [9]. This produces an intermediate artifact that can be inspected independently of the final image. A valid Unicode sequence, an appropriate transliteration, and a readable visual rendering are related requirements, but they are not interchangeable tests.

III. Proposed methodology

A. Input normalization and transliteration

The pipeline receives two primary inputs: Latin text and a selected material style. We first normalize the text to the transliteration convention expected by the conversion path. One implemented mapping changes “wa” to “va” when preparing input for the ISO-based Aksharamukha conversion. This mapping is a practical interface convention; it should not be interpreted as a complete resolution of regional spelling, pronunciation, or lexical ambiguity.

We then convert the normalized string into Javanese Unicode and prepare it for rendering with Noto Sans Javanese. The implementation checks that the font asset is available before drawing. This prevents a missing font from silently turning the spatial guide into replacement characters. Keeping the normalized Latin string, Unicode sequence, and rendered guide conceptually separate also makes it possible to identify whether an error originates before or after image synthesis.

B. Wrapping, sizing, and centering the glyph guide

We render white glyphs on a dark-gray canvas with RGB value (20, 20, 20). The inference implementation uses 1024 × 680 pixels. The layout routine starts from a 220-pixel font size, measures the complete text block, and reduces the size in five-pixel increments while the block exceeds the available width or height. The documented bounds are w − 250 horizontally and h − 200 vertically. Reserving this space is particularly important for marks that extend beyond the main letter body.

Our wrapping helper uses a word-count limit, described as a maximum of three words per line. After wrapping, the block is centered using its measured bounds rather than by stretching the glyphs to fill the canvas. In geometric terms, for measured bounds (x₀, y₀, x₁, y₁), a centered draw origin can be expressed as x = (W − (x₁ − x₀))/2 − x₀ and y = (H − (y₁ − y₀))/2 − y₀. These expressions explain the bounding-box centering principle; they do not imply an independently measured perceptual optimum.

The data-construction plan records 1024 × 682 pixels, whereas the implemented inference path records 1024 × 680. We retain that distinction rather than treating the dimensions as identical. A reproducible implementation must fix the canvas size explicitly and preserve it through guide generation and inference.

C. Parallel structural guidance

We extract the Canny map using lower and upper thresholds of 100 and 200. Canny detection combines gradient processing, non-maximum suppression, and hysteresis thresholding [10]. Here, its output describes the external and internal boundaries of the rendered writing, including small attached components. The edge guide is intended to discourage shape drift when the material prompt introduces complex texture.

For the second branch, the implementation converts the mask into a single-channel image with mask.convert("L"). We feed this intensity guide to a depth-conditioned ControlNet. It is a mask-derived proxy, not a measured depth map or an independently estimated physical surface. Its useful distinction is between the writing region and its background. Relief-like shading in the output is a synthesized visual effect, not evidence of recovered three-dimensional geometry.

D. Prompt conditioning and image synthesis

The generation stage combines a material prompt, a negative prompt, the two spatial guides, and the trained LoRA adapter. Material prompts describe appearance such as reflective gold, weathered stone, ink, carved wood, or neon illumination. Negative prompting discourages unwanted background elements and pseudo-characters. These text conditions complement the spatial guide; they do not provide an orthographic guarantee.

Classifier-free guidance changes the balance between conditional and unconditional denoising predictions [11]. Our preferred configuration uses a guidance scale of 8.5, Canny conditioning of 1.1, depth-branch conditioning of 0.8, and LoRA strength of 0.85. We use 50 inference steps with DPMSolverMultistepScheduler. DPM-Solver++ provides efficient high-order methods for guided sampling [12], and the Diffusers implementation exposes scheduler choices that must be specified for exact reproduction [13].

The normal-condition experiment also records an image-to-image denoising-strength range of 0.75–0.80. We distinguish this recorded range from the fixed settings of the six-configuration comparison, where strength is not separately tabulated. After denoising, the VAE decoder converts the latent representation into an image. The application displays the output and supports PNG export for subsequent inspection and creative use.

IV. Dataset and experimental setup

A. Construction of the synthetic collection

We construct the dataset through four linked artifacts: a Latin input, a rendered script mask, a stylized target image, and a caption. The collection was developed over four weeks during February–March 2026. It contains 70 image–caption pairs across five material categories. We use Sora AI to produce stylized targets from the script guides, then pair each selected target with descriptive text using the trigger akshara_style. This process supplies appearance examples for adaptation without relying on a large existing collection of material-styled Javanese calligraphy.

The caption files share their basenames with their image files so the training loader can associate the correct description with each target. Category folders organize the material styles. The documented directory convention includes repetition prefixes such as 10_akshara_gold for the Kohya_ss data-loading workflow. A repetition prefix should not be confused with a complete training-step calculation: batch size, accumulation, epochs, and the training configuration also affect optimization.

Royal Gold, Ancient Stone, Manuscript, and Wood Carving each contribute 15 pairs; Modern Neon contributes 10. These are training-collection counts, not a held-out test-set size. The collection is intentionally small and uneven. We therefore treat it as a focused adaptation resource, rather than evidence of broad coverage of Javanese words, spelling conventions, or all combinations of dependent signs.

B. Implementation and computing environment

We use Python for preprocessing and orchestration, Aksharamukha for transliteration, Pillow for layout, OpenCV for edge extraction, Hugging Face Diffusers for inference, and Kohya_ss for LoRA training. Hugging Face Hub supplies model artifacts. Google Drive provides persistent dataset and checkpoint storage across temporary notebook sessions. The development setup separates lightweight text and mask processing from GPU-intensive adaptation and diffusion.

The experimental toolchain uses Google Colab and Kaggle notebooks, with local development on an Apple M2 system. T4, P100, and A100 devices appear in the computing plan as possible cloud resources; we do not assign an unrecorded device or memory measurement to an individual result. Similarly, the documentation discusses multiple training and inference environments without providing a complete run-by-run hardware log. The study evaluates output behavior, not a benchmark of throughput or GPU efficiency.

C. Checkpoint selection

We inspect Wood Carving outputs across checkpoints from 1,000 to 6,000 training steps. Early outputs contain decorative growth and unstable letter structure. Intermediate outputs begin to organize the text and localize the wood texture, but some edges and small components remain weak. The final group offers the strongest qualitative balance in the inspected examples. We select the 6,000-step checkpoint for the subsequent configuration study.

We use these observations as a checkpoint-selection rationale. Without a numerical learning curve, a held-out score, or repeated training runs, visual preference does not establish convergence or exclude overfitting. This distinction matters when carrying a promising research checkpoint into a reusable GAKSARA workflow.

D. Six configurations of guide and synthesis parameters

We compare six settings that vary guide polarity, background intensity, ControlNet strength, LoRA strength, guidance, and sampling steps. Configurations A–C use the edge branch alone; D–F introduce the second branch. The purpose is to identify an operational balance between structural retention and material expression. Several variables change between runs, so the experiment is an iterative configuration comparison rather than a single-factor causal ablation.

V. Results and discussion

A. Balancing structure and appearance

Configuration A produces extraneous background characters under a black-on-white guide. Reversing the guide in B improves the separation of the writing from the background, but the material appears flat. In C, a dark-gray background and adjusted synthesis settings improve integration. D adds dual conditioning but produces rigid forms. E strengthens stylistic freedom while reducing both control scales, accompanied by unwanted background characters. We select F after restoring stronger spatial guidance and adjusting guidance and step count.

This progression shows why legibility and visual quality require separate inspection. A clear outline can still look pasted onto its background, while an appealing texture can conceal altered marks. The configuration comparison helps identify a useful region for our prototype, but it does not isolate the effect of any single setting. In particular, the differences between C and F cannot be attributed exclusively to the second branch because other parameters also change.

The Ancient Stone “prambanan” output shown in the hero is our representative normal-condition result. We inspect its cakra modifier and vertically arranged pasangan form alongside the stone texture. In the selected example, these structural components remain visually identifiable while the output develops raised contours, weathering, and moss-like detail. These observations support the feasibility of the hybrid approach for the demonstrated case; they do not establish correctness for every possible input.

B. Material-specific results

Across the five styles, we obtain visibly different treatments: specular metal in Royal Gold, eroded relief in Ancient Stone, ink and substrate texture in Manuscript, directional grain in Wood Carving, and luminous edges in Modern Neon. The examples use different strings and layouts. They demonstrate the scope of visual treatments available in the experiment rather than a matched ranking of styles on a common test input.

Royal GoldInput: borobudur

Ancient StoneInput: setya mulyo

ManuscriptInput: sabar subur

Wood CarvingInput: gemi nastiti ati ati

Modern Neon

These outputs also clarify the division of labor in the system. A material effect is valuable only when the reader can still recognize the intended writing. We therefore inspect the main body, dependent marks, spacing, and background separately instead of treating texture richness as a proxy for orthographic quality. For GAKSARA, this motivates a review workflow in which users can compare the generated asset with its script guide.

C. Multi-line text and layout pressure

We test a longer input, “bocah seger badan sehat gemi nastiti ati ati rahayu,” to exercise wrapping and font reduction. The resulting image contains three lines. As text density increases, the layout stage must reduce the glyph size and retain space around vertically extending components. The example demonstrates a working multi-line path, but it also exposes the need to make wrapping rules and fit decisions explicit.

The documented three-word wrapping limit and the displayed three-line sentence do not, by themselves, specify the exact tokenization and line-breaking decisions. We therefore do not generalize this single example into a universal text-length limit. A robust implementation should expose the rendered guide before generation and reject a layout that becomes too small or crowded for reliable inspection.

D. Extreme parameter settings

We test three Ancient Stone settings to understand failure behavior. High LoRA strength with weak control spreads material features outside the intended region. Low LoRA strength with strong control preserves a rigid outline but suppresses the desired material effect. Raising both strengths produces dense texture and excessive contrast. The original stress configuration records a single control scale without a separate depth value, so these examples do not constitute a fully specified two-branch sweep.

Over-stylizationLoRA 1.5 · Control 0.5

Structural dominanceLoRA 0.2 · Control 1.2

Over-conditioningLoRA 1.5 · Control 1.2

E. What the evaluation establishes

Our evaluation is qualitative and example-based. We identify a preferred configuration, document material variation, and expose concrete failure modes. The initial experimental notes use “zero-typo” and “0% deformation” for selected outputs. We interpret those terms as visual assessments of the displayed examples, not as a measured general error rate. We have not established a character-error denominator, independent reader agreement, confidence interval, or repeated-seed benchmark. A general zero-error rate cannot be inferred from this evaluation.

The strongest supported conclusion is that explicit script preparation and spatial conditioning can produce useful material-styled Javanese imagery in the demonstrated workflow. The evidence does not establish benchmark superiority, complete orthographic coverage, or production reliability. These boundaries guide our next experiments rather than reducing the practical value of documenting the implemented system.

VI. GAKSARA: from research workflow to application

A. User interaction and prototype behavior

We bring the generation workflow into GAKSARA through a web interface that connects text input, style selection, and output review. The interaction begins with an introduction to the script and the generation process. The main workspace then accepts Latin text and offers the five material categories. After generation, the interface presents the image on a dedicated result canvas and supports exporting the generated PNG.

The original prototype connects its front end to cloud inference. This separation allows the browser to handle the interaction while the GPU runtime performs diffusion. It also creates operational dependencies: a result cannot be generated when the notebook session has ended, the model is not loaded, or the tunnel is unavailable. We treat these as engineering requirements for the application, not as details that a more polished interface can eliminate.

B. Adaptation at Data Sorcerers

For GAKSARA at Data Sorcerers, we retain the research principle that script structure should be inspectable before and after synthesis. The product direction is a cultural creative tool, with material selection built around a prepared script rather than an unconstrained request to invent lettering. This article documents the research foundation of that direction; adapting it to GAKSARA does not create a new benchmark or imply that the original experiments have been repeated at production scale.

Potential uses include educational graphics, cultural-event communication, contemporary calligraphy, and design exploration. These are application opportunities, not outcomes measured in a user study. The PNG is a visual asset rather than a replacement for the underlying text. A preservation-oriented application should retain the original input, normalized form, Unicode sequence, selected style, and generation settings alongside the image so that users can understand how an asset was produced.

C. Engineering roadmap

The prototype uses notebook resources and tunneling, which are suitable for experimentation but expose session and availability limits. Our next engineering direction is a dedicated GPU service with a defined inference API, request lifecycle, and persistent artifacts. The development plan identifies FastAPI and Docker as components for that transition. They are planned infrastructure choices, not evidence that high availability, concurrent throughput, or latency targets have already been achieved.

We also plan to expand material coverage through additional curated data and adapters. Candidate directions include embossed metal, holographic treatments, watercolor ink, and brush calligraphy. Each extension requires its own inspection of glyph retention and failure modes. A larger menu of styles would not, on its own, demonstrate more reliable writing.

VII. Limitations and future evaluation

A. Data coverage and synthetic-target bias

Seventy pairs provide a focused adaptation set, but they do not represent the full variation of Javanese writing. The unequal category counts, limited lexical coverage, and use of generated targets constrain what can be concluded. A synthetic target may already contain an error, so visual consistency across training examples cannot substitute for script expertise. The current record does not establish a lexical train/validation/test split or show that repeated words across styles are isolated between partitions.

B. Reproducibility of the training and inference record

The next release of the experimental record should include exact base-model and ControlNet revisions, LoRA rank and target modules, optimizer settings, learning rate, batch size, accumulation, data repetitions, random seeds, and complete prompts. We also need the scheduler algorithm and solver order, which are not uniquely determined by naming its class. The current documentation does not provide all of these values, and we do not fill the gaps with assumed defaults.

A dataset manifest should connect each text string to its normalized form, Unicode output, mask, material target, and caption. It should also resolve the 680-versus-682-pixel canvas difference and record the font file and shaping environment. These details are particularly important for small marks and line placement: a change in rendering can alter the control image before the diffusion process even begins.

C. A stronger evaluation protocol

We propose a held-out evaluation that separates lexical content across all material variants. Text-only generation, Canny-only guidance, intensity-proxy-only guidance, and dual guidance should be compared with matched prompts, seeds, and sampling budgets. Reporting every run would make failure frequency visible and avoid selecting only attractive outputs. These are planned comparisons, not results of the present study.

Expert readers should assess the Unicode conversion independently of the image. Image assessment should then measure missing or added components, retention of dependent signs, background pseudo-characters, readability, and material consistency under a stated scoring convention. Character or grapheme error measures require an explicit reference transcription and denominator. Multiple reviewers, agreement reporting, and repeated seeds would help distinguish systematic limitations from individual generation failures.

D. Cultural use and operating limits

We position GAKSARA as an aid to creative cultural expression, with human review retained for meaningful public use. A generated stone texture must not be presented as an authentic archaeological inscription. Likewise, contemporary manuscript styling is not a restoration of a historical document. Keeping text provenance and transformation settings makes this distinction easier to maintain.

The present work does not provide measured latency, concurrent-user capacity, uptime, or a longitudinal adoption study. Future application evaluation should therefore assess linguistic quality and the reliability of the complete interaction, including failed generation requests and export behavior. We regard those as separate questions from whether a selected image is visually convincing.

VIII. Conclusion

We have developed a hybrid workflow that separates Javanese text preparation from generative material synthesis and adapted its research foundation for GAKSARA at Data Sorcerers. The pipeline combines normalization and transliteration, adaptive glyph layout, dual ControlNet guidance, and LoRA-adapted Stable Diffusion. Our 70-pair collection, checkpoint study, six configurations, and stress tests provide a concrete account of how structural control and material expression interact.

The preferred configuration produces useful examples across five styles, while extreme settings expose spillover, flattened material, and excessive contrast. Integrating these stages into a web prototype makes the workflow accessible through text entry, style selection, visual review, and image export. The next step is to make the experimental record reproducible, validate orthography through independent readers, and build dependable serving infrastructure. Preserving the writing remains the central requirement as GAKSARA develops beyond the initial prototype.

References

1. [1]

   Z. A. Afham, “Pengembangan content generator model berbasis Stabe Diffusion dan ControlNet untuk pelestarian digital Aksara Jawa,” research documentation, 2026. Original experimental record underlying this GAKSARA article.

2. [2]

   The Unicode Consortium, “Javanese: Range A980–A9DF,” Unicode character code charts. Accessed: Oct. 31, 2026. Source ↗

3. [3]

   J. Chen, Y. Huang, T. Lv, L. Cui, Q. Chen, and F. Wei, “TextDiffuser: Diffusion models as text painters,” in Advances in Neural Information Processing Systems, 2023. arXiv:2305.10855. Source ↗

4. [4]

   J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in Advances in Neural Information Processing Systems, 2020. arXiv:2006.11239. Source ↗

5. [5]

   R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022. arXiv:2112.10752. Source ↗

6. [6]

   L. Zhang, A. Rao, and M. Agrawala, “Adding conditional control to text-to-image diffusion models,” in Proc. IEEE/CVF International Conference on Computer Vision, 2023, pp. 3836–3847. arXiv:2302.05543. Source ↗

7. [7]

   E. J. Hu et al., “LoRA: Low-rank adaptation of large language models,” arXiv:2106.09685, 2021. Source ↗

8. [8]

   V. Rajan, “Aksharamukha Python library,” software repository. Accessed: Oct. 31, 2026. Source ↗

9. [9]

   Pillow contributors, “ImageDraw module,” Pillow documentation. Accessed: Oct. 31, 2026. Source ↗

10. [10]

    OpenCV contributors, “Canny edge detection,” OpenCV documentation. Accessed: Oct. 31, 2026. Source ↗

11. [11]

    J. Ho and T. Salimans, “Classifier-free diffusion guidance,” arXiv:2207.12598, 2022. Source ↗

12. [12]

    C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li, and J. Zhu, “DPM-Solver++: Fast solver for guided sampling of diffusion probabilistic models,” arXiv:2211.01095, 2022. Source ↗

13. [13]

    Hugging Face, “DPMSolverMultistepScheduler,” Diffusers documentation. Accessed: Oct. 31, 2026. Source ↗