<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[ProTechSC Academy]]></title><description><![CDATA[ProTechSC Academy]]></description><link>https://protechscacademy.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Wed, 18 Nov 2026 14:56:24 GMT</lastBuildDate><atom:link href="https://protechscacademy.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[A 3B language model can't plan robot moves — but it can tell a planner what to do (0% → 76%)]]></title><description><![CDATA[LLM+P on PlanBench Blocksworld with qwen2.5:3b, LangGraph, PDDL, pyperplan and a simulated PyBullet arm]]></description><link>https://protechscacademy.hashnode.dev/a-3b-language-model-can-t-plan-robot-moves-but-it-can-tell-a-planner-what-to-do-0-76</link><guid isPermaLink="true">https://protechscacademy.hashnode.dev/a-3b-language-model-can-t-plan-robot-moves-but-it-can-tell-a-planner-what-to-do-0-76</guid><category><![CDATA[llm]]></category><category><![CDATA[Python]]></category><category><![CDATA[robotics]]></category><category><![CDATA[Artificial Intelligence]]></category><dc:creator><![CDATA[purushothaman sinivasan]]></dc:creator><pubDate>Mon, 16 Nov 2026 17:13:32 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6af87beba59ec55fc176a005/e6cb03a2-9e1f-451f-b816-02168d7543ce.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Ask a small language model to write the step-by-step plan for stacking blocks, and it fails on <strong>every single task</strong>. Ask the same model only to <em>translate the instruction into a goal</em>, and let a classical planner do the planning, and <strong>76%</strong> of the plans are correct. This post walks through that experiment and the one prompt change that made the difference.</p>
<p>All plans run only on a <strong>simulated</strong> arm in PyBullet. The executor never moves a real robot and never runs a plan that its own validator rejects.</p>
<h2>The task</h2>
<p>We used <strong>PlanBench Blocksworld</strong>, a standard benchmark for testing whether language models can plan. There are 50 instances with 3–5 blocks, and each instruction is given in <strong>two phrasings</strong>: the original PlanBench sentence and a natural paraphrase. That makes 100 evaluation items.</p>
<p>An instruction looks like this: <em>"My goal is to have the red block on top of the blue block and the blue block on top of the orange block."</em></p>
<p>The main metric is the <strong>goal-correct plan rate</strong>: the plan must be valid (every action's preconditions hold) <strong>and</strong> must reach the gold goal.</p>
<h2>Six configurations, same 100 items</h2>
<table>
<thead>
<tr>
<th>Configuration</th>
<th>What it does</th>
<th>Goal-correct</th>
</tr>
</thead>
<tbody><tr>
<td>Random walk</td>
<td>random legal moves until the goal holds</td>
<td>12%</td>
</tr>
<tr>
<td>Rule parser + planner</td>
<td>template parser for the PlanBench sentence → PDDL goal → planner</td>
<td>50%</td>
</tr>
<tr>
<td><strong>LLM direct plan</strong></td>
<td>qwen2.5:3b writes the action sequence itself</td>
<td><strong>0%</strong></td>
</tr>
<tr>
<td><strong>LLM+P agent</strong></td>
<td>qwen2.5:3b writes only the <strong>PDDL goal</strong>; pyperplan plans; validator checks</td>
<td><strong>76%</strong></td>
</tr>
<tr>
<td>Agent without repair</td>
<td>same, without the one repair round</td>
<td>74%</td>
</tr>
<tr>
<td>Gold goal + planner</td>
<td>hand-written goal → planner (upper bound)</td>
<td>100%</td>
</tr>
</tbody></table>
<p><img src="https://dev-to-uploads.s3.us-east-2.amazonaws.com/uploads/articles/cdh7cetp9f4k6efnlrkm.png" alt="Valid, goal-correct and PyBullet execution rates, 95% bootstrap CIs" />
<em>Valid, goal-correct and PyBullet execution rates, 95% bootstrap CIs</em></p>
<p>All <strong>100 direct plans</strong> broke a precondition, for example picking up a block that had another block on top of it. The agent beat direct planning on 76 items and lost on none (McNemar p = 2.6 × 10⁻²³). It averaged <strong>1.14 LLM calls per item</strong>, and <strong>every correct plan was optimal in length</strong>, because the planner, not the LLM, chooses the moves.</p>
<p>The rule parser shows why a language model is still useful: it scores <strong>100% on PlanBench sentences but 0% on paraphrases</strong>. The agent handled both phrasings (74% and 78%).</p>
<h2>The architecture (LLM+P)</h2>
<ol>
<li><strong>Scene</strong>: the LangGraph planner agent asks the robot executor for the current block layout over the <strong>Agent2Agent (A2A)</strong> protocol.</li>
<li><strong>Translate</strong>: qwen2.5:3b (served by Ollama, temperature 0) turns the instruction into a <strong>PDDL goal</strong>, using a few examples.</li>
<li><strong>Check</strong>: a goal checker rejects unknown blocks, cycles (A on B on A) and goals that are already true.</li>
<li><strong>Plan</strong>: pyperplan (A* with the hFF heuristic) finds an optimal plan.</li>
<li><strong>Validate</strong>: our own STRIPS validator replays the plan. If anything fails, the error is fed back to the LLM for <strong>one repair</strong>.</li>
<li><strong>Execute</strong>: the A2A robot executor <strong>validates again</strong> and runs the plan on a simulated arm in PyBullet.</li>
</ol>
<h2>The prompt change that mattered: use colour names</h2>
<p>We designed the prompt on 8 separate DEV items only, never on the evaluation set:</p>
<table>
<thead>
<tr>
<th>Version</th>
<th>Change</th>
<th>DEV goal-correct</th>
</tr>
</thead>
<tbody><tr>
<td>v1</td>
<td>PDDL objects named a, b, c, d (as in PlanBench) + 3 examples</td>
<td>4/8</td>
</tr>
<tr>
<td>v2</td>
<td>v1 + tower rule + cycle check + 6 examples</td>
<td>3/8</td>
</tr>
<tr>
<td><strong>v3</strong></td>
<td><strong>colour names (red, blue, …) as PDDL object names</strong>, mapped back to letters for the planner</td>
<td><strong>7/8</strong></td>
</tr>
</tbody></table>
<p>The 3B model kept mixing up <em>which letter belongs to which colour</em>, and adding more examples made it worse. Letting it write <code>(on red blue)</code> instead of <code>(on a b)</code> removed that mapping step entirely. This is a useful general lesson: <strong>remove indirection the model has to track</strong>, rather than adding examples.</p>
<h2>Where the agent still fails</h2>
<p>The 24 failures are all <strong>translation errors</strong>, never planning errors:</p>
<ul>
<li>19 wrong or swapped blocks</li>
<li>3 missing goal facts</li>
<li>1 unsatisfiable goal</li>
<li>1 parse failure</li>
</ul>
<p><img src="https://dev-to-uploads.s3.us-east-2.amazonaws.com/uploads/articles/a8p2zwcjv4ph7cttpojp.png" alt="Failure categories per configuration" />
<em>Failure categories per configuration</em></p>
<p>The repair round fixed only 2 of the 14 items it was triggered on (76% vs 74%, p = 0.5, not significant). The repair <em>did</em> convert many invalid plans into valid ones (97% valid vs 86%), but a valid plan for the wrong goal is still wrong.</p>
<h2>Limitations</h2>
<ul>
<li>100 items gives confidence intervals of about ±9 points, and the two phrasings of the same instance are correlated, so the intervals are somewhat optimistic.</li>
<li>Blocksworld is a toy domain. The Logistics domain was not done.</li>
<li>One small model (3B). A larger model would probably close part of the gap to the 100% upper bound.</li>
</ul>
<h2>Takeaways for agent builders</h2>
<ol>
<li><strong>Don't ask an LLM to do what a solver does reliably.</strong> Ask it to produce the solver's input.</li>
<li><strong>Validate twice</strong>: in the planner and again in the executor.</li>
<li><strong>Measure against a non-LLM baseline</strong> and an upper bound, not just against "the LLM alone".</li>
<li><strong>Tune prompts on a separate DEV set</strong>, and report what didn't work.</li>
</ol>
<hr />
<p><em>Full project (LangGraph agent, A2A robot executor, PyBullet simulation, MCP tool server, web app with plan traces, report): <a href="https://www.protechscacademy.com/projects/agent-development/language-to-plan-robot-task-planner-with-llm-and-pddl">Language-to-plan robot task planner — ProTechSC Academy</a>.</em></p>
<p><em>We guide final-year and PhD research projects in engineering, AI and AI agents. WhatsApp +91 97907 09424.</em></p>
]]></content:encoded></item><item><title><![CDATA[Optimising an airfoil with a neural-network surrogate — and checking every answer with a real solver]]></title><description><![CDATA[2,400 airfoils, a deep ensemble, a genetic algorithm and active learning: NACA 2412 mean L/D 77 to 114 at Re 1 million]]></description><link>https://protechscacademy.hashnode.dev/optimising-an-airfoil-with-a-neural-network-surrogate-and-checking-every-answer-with-a-real-solver</link><guid isPermaLink="true">https://protechscacademy.hashnode.dev/optimising-an-airfoil-with-a-neural-network-surrogate-and-checking-every-answer-with-a-real-solver</guid><category><![CDATA[Machine Learning]]></category><category><![CDATA[Python]]></category><dc:creator><![CDATA[purushothaman sinivasan]]></dc:creator><pubDate>Sat, 14 Nov 2026 15:33:50 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6af87beba59ec55fc176a005/488d3107-36cc-4189-9daa-59ef2d04a8cb.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Shape optimisation needs thousands of aerodynamic evaluations. If each one is a CFD run, the optimiser becomes the bottleneck. The usual answer is a <strong>surrogate model</strong>: train a neural network on solver results, then let the optimiser query the network instead.</p>
<p>The risk is that <strong>an optimiser is very good at finding where a surrogate is wrong</strong>. This article shows how we built an airfoil optimisation loop that uses AI for speed, while every answer it gives is verified by the physics solver.</p>
<h2>1. A compact shape description</h2>
<p>We describe the airfoil with <strong>CST (Kulfan) parameters</strong>: 6 Bernstein weights per surface, 12 numbers in total. CST gives smooth shapes with a round nose and a sharp trailing edge. NACA 2412 is fitted almost exactly, so it serves as the baseline. The design space is a ±0.09 box around that fit, with thickness from 6 to 20%.</p>
<h2>2. A fast viscous solver</h2>
<p>We wanted thousands of runs on a laptop, so instead of RANS CFD we wrote a classical <strong>viscous-inviscid</strong> solver:</p>
<ul>
<li>a linear-vortex <strong>panel method</strong> (160 panels, Kutta condition)</li>
<li><strong>Thwaites</strong> laminar boundary layer, <strong>Michel</strong> transition, <strong>Head</strong> turbulent method</li>
<li>Squire-Young drag and three displacement-thickness coupling passes</li>
</ul>
<p>We validated it against thin-airfoil theory. The lift slope was 6.87 vs 6.86 per radian, and the zero-lift angle was −2.15° vs −2.08°. We also checked panel independence. The same shapes are exported with ANSYS Fluent journals and a Gmsh mesh script for a later RANS check.</p>
<p><strong>Data set:</strong> 2,400 Latin-hypercube airfoils × 6 angles at Re = 1 × 10⁶, giving 14,400 polars in 160 seconds on 6 CPU cores.</p>
<h2>3. The AI surrogate: a deep ensemble</h2>
<p>We trained <strong>five independent MLPs</strong> (13 inputs → 3 × 128 SiLU → Cl, log₁₀Cd, Cm). The ensemble mean is the prediction, and <strong>the spread between the five networks is the uncertainty</strong>. A separate gradient-boosting classifier predicts trailing-edge separation.</p>
<p>On 360 airfoils it never saw (we split the data by airfoil, not by row), the results were:</p>
<table>
<thead>
<tr>
<th>Quantity</th>
<th>Error</th>
</tr>
</thead>
<tbody><tr>
<td>Lift coefficient</td>
<td>R² = 0.9998</td>
</tr>
<tr>
<td>Drag coefficient</td>
<td>1.6%</td>
</tr>
<tr>
<td>L/D</td>
<td>1.7% (gradient boosting 3.8%, quadratic response surface 5.5%)</td>
</tr>
<tr>
<td>Separation</td>
<td>AUC 0.997</td>
</tr>
<tr>
<td>Speed</td>
<td>1,000 predictions in 17 ms</td>
</tr>
</tbody></table>
<p><img src="https://dev-to-uploads.s3.us-east-2.amazonaws.com/uploads/articles/96v8o9c511kmft2bmp3y.png" alt="Predicted vs solver values on unseen airfoils" />
<em>Predicted vs solver values on unseen airfoils</em></p>
<h2>4. Optimise on the network, but cautiously</h2>
<p>A real-coded genetic algorithm (80 designs × 120 generations) maximises the mean L/D at 2°, 4° and 6°. It optimises <strong>minus one ensemble standard deviation</strong>, a lower confidence bound. This penalises designs where the five networks disagree, which is exactly where the surrogate is guessing.</p>
<p>The constraints were:</p>
<ul>
<li>thickness ≥ 12%</li>
<li>trailing-edge thickness at 90% chord ≥ 1.5%</li>
<li>pitching moment Cm ≥ −0.10</li>
<li>no separation</li>
</ul>
<h2>5. Active learning: let the solver correct the network</h2>
<p>After each GA run, the 24 best candidates are <strong>re-run with the solver</strong>. The results are added to the training set, and the ensemble is fine-tuned. Over three rounds, the verified L/D rose from 112.6 to 112.8, then 113.0, then <strong>114.3</strong>. The surrogate's error at the optimum fell to 0.3–1.2%.</p>
<p>As a fairness check, we also ran a GA that calls the solver directly, with the same number of solver calls. It reached 112.9, slightly below the surrogate loop.</p>
<h2>Results (all verified with the solver)</h2>
<table>
<thead>
<tr>
<th></th>
<th>NACA 2412</th>
<th>Optimised (M1-OPT)</th>
</tr>
</thead>
<tbody><tr>
<td>Mean L/D at 2/4/6°</td>
<td>77.2</td>
<td><strong>114.3</strong></td>
</tr>
<tr>
<td>Maximum L/D</td>
<td>87.6</td>
<td>128.5</td>
</tr>
<tr>
<td>L/D at equal lift, Cl = 0.7</td>
<td>80.7</td>
<td>92.1 (+14%)</td>
</tr>
<tr>
<td>L/D at equal lift, Cl = 0.9</td>
<td>86.9</td>
<td>112.6 (+30%)</td>
</tr>
<tr>
<td>Thickness</td>
<td>12%</td>
<td>12%</td>
</tr>
</tbody></table>
<p><img src="https://dev-to-uploads.s3.us-east-2.amazonaws.com/uploads/articles/qiy0m0q9snjpn6tne2ny.png" alt="Viscous polars of the baseline, the AI-optimised airfoil and the direct GA" />
<em>Viscous polars of the baseline, the AI-optimised airfoil and the direct GA</em></p>
<p><strong>Comparing at equal lift matters.</strong> Part of the gain at a fixed angle comes from the optimum's extra camber. At equal lift the improvement is 14–30%, which is still large.</p>
<p>The gain also holds off-design. Mean L/D was 91.5 vs 68.5 at Re = 5 × 10⁵, and 150 vs 93 at Re = 3 × 10⁶.</p>
<p><img src="https://dev-to-uploads.s3.us-east-2.amazonaws.com/uploads/articles/qtbz55jnl4gt8rkpd6mu.png" alt="Baseline and optimised shapes" />
<em>Baseline and optimised shapes</em></p>
<h2>Limitations</h2>
<ul>
<li>At low lift (Cl ≈ 0.5), the optimum is <strong>4% worse</strong> than NACA 2412. It is a cruise/climb section, not a universal improvement.</li>
<li>The pitching moment sits exactly at its limit (−0.10), so the optimiser used everything it was allowed.</li>
<li>A panel/boundary-layer solver is not RANS CFD. A Fluent k-ω SST check of the final shape is the natural next step, and the journals for it are included.</li>
</ul>
<h2>What transfers to other design problems</h2>
<ol>
<li><strong>Use an ensemble so the surrogate knows when it is unsure,</strong> and optimise a lower confidence bound.</li>
<li><strong>Never report a surrogate optimum without re-running the real solver.</strong></li>
<li><strong>Active learning</strong> puts the expensive solver calls where they matter.</li>
<li><strong>Compare fairly:</strong> at equal lift, and against a direct optimiser on the same budget.</li>
</ol>
<p>Once trained, the surrogate can be reused. New requirements, such as 14% thickness, a stricter moment limit or climb angles, were each optimised in about 5 seconds with only 18 solver calls.</p>
<hr />
<p><em>Full project description, report and video: <a href="https://www.protechscacademy.com/projects/mechanical-engineering/ai-based-aerodynamic-optimization-of-an-airfoil">AI-based aerodynamic optimisation of an airfoil — ProTechSC Academy</a>. Video walkthrough: <a href="https://youtu.be/O1sr7P9WCcE">youtu.be/O1sr7P9WCcE</a>.</em></p>
<p><em>We guide final-year and PhD research projects in engineering and AI. WhatsApp +91 97907 09424.</em></p>
]]></content:encoded></item></channel></rss>