<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Yafei&apos;s Archive</title><description>Research, Robotics, and Reflections</description><link>https://www.yafei.dev/</link><language>en</language><item><title>A Quick Hack for Texturing a 3D Model</title><link>https://www.yafei.dev/posts/2026-03-16-a-quick-hack-for-texturing/</link><guid isPermaLink="true">https://www.yafei.dev/posts/2026-03-16-a-quick-hack-for-texturing/</guid><description>Quick experiment generating textures for a RealSense D435i 3D model.</description><pubDate>Mon, 16 Mar 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;import ImageWrapper from &quot;../../../../components/misc/MarkdownImageWrapper.astro&quot;;
import ImageRealSenseGenerated from &quot;./realsense-generated.png&quot;;
import ImageRealSense from &quot;./realsense-d435i.png&quot;;
import ImageRealSense2 from &quot;./realsense-d435i-2.png&quot;;&lt;/p&gt;
&lt;p&gt;Although the CAD model (SolidWorks) for the RealSense D435i camera is available in the official documentation, I could not find a textured 3D model suitable for simulation or rendering.&lt;/p&gt;
&lt;p&gt;So I tried a quick workaround: generating views of each side of the camera using images of the device found online, with the help of Nano Banana Pro.&lt;/p&gt;
&lt;p&gt;See one of the generated views below:&lt;/p&gt;
&lt;p&gt;&amp;lt;ImageWrapper
src={ImageRealSenseGenerated.src}
alt=&quot;Generated bottom view of RealSense D435i&quot;
caption=&quot;Generated bottom view of RealSense D435i using Nano Banana Pro&quot;
class=&quot;w-full h-full&quot;
/&amp;gt;&lt;/p&gt;
&lt;p&gt;These generated views can then be used as textures during UV unwrapping (e.g., using Project from View). By simply aligning the vertices with the image, the model can quickly obtain a usable texture.&lt;/p&gt;
&lt;p&gt;This is far from perfect, but given the minimal cost (except for the Google One subscription fee), the results are surprisingly usable.&lt;/p&gt;
&lt;p&gt;&amp;lt;ImageWrapper
src={ImageRealSense.src}
alt=&quot;Shaded RealSense D435i model in Blender&quot;
caption=&quot;Shaded RealSense D435i model in Blender&quot;
class=&quot;w-full h-full&quot;
/&amp;gt;&lt;/p&gt;
&lt;p&gt;&amp;lt;ImageWrapper
src={ImageRealSense2.src}
alt=&quot;Shaded RealSense D435i model in Blender&quot;
caption=&quot;Shaded RealSense D435i model in Blender&quot;
class=&quot;w-full h-full&quot;
/&amp;gt;&lt;/p&gt;
&lt;hr /&gt;
&lt;p&gt;Written by Yafei Ou and reviewed for grammar and conceptual accuracy by ChatGPT (2026-03-16).&lt;br /&gt;
© 2026 Yafei Ou. All rights reserved.&lt;/p&gt;
</content:encoded></item><item><title>一个快速为 3D 模型添加纹理的小技巧</title><link>https://www.yafei.dev/posts/zh-cn/2026-03-16-a-quick-hack-for-texturing/</link><guid isPermaLink="true">https://www.yafei.dev/posts/zh-cn/2026-03-16-a-quick-hack-for-texturing/</guid><description>为 RealSense D435i 3D 模型快速生成纹理。</description><pubDate>Mon, 16 Mar 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;import ImageWrapper from &quot;../../../../components/misc/MarkdownImageWrapper.astro&quot;;
import ImageRealSenseGenerated from &quot;./realsense-generated.png&quot;;
import ImageRealSense from &quot;./realsense-d435i-2.png&quot;;
import ImageRealSense2 from &quot;./realsense-d435i.png&quot;;&lt;/p&gt;
&lt;p&gt;尽管在官方文档中可以找到 RealSense D435i 相机的 CAD 模型（SolidWorks），但是我没找到能够用于仿真或渲染的带纹理三维模型。&lt;/p&gt;
&lt;p&gt;因此，我尝试了一个简单的变通方法：借助 Nano Banana Pro，利用网上找到的设备图片生成相机各个侧面的视图。&lt;/p&gt;
&lt;p&gt;下面是生成视图中的一个示例：&lt;/p&gt;
&lt;p&gt;&amp;lt;ImageWrapper
src={ImageRealSenseGenerated.src}
alt=&quot;Generated bottom view of RealSense D435i&quot;
caption=&quot;Nano Banana Pro 生成的 RealSense D435i 底部视图&quot;
class=&quot;w-full h-full&quot;
/&amp;gt;&lt;/p&gt;
&lt;p&gt;这些生成的视图可以在进行 UV 展开时用作纹理（例如使用“从视图投影”）。只需要将模型的顶点与图像对齐，就可以快速为模型赋予一个可用的纹理。&lt;/p&gt;
&lt;p&gt;虽说远非完美，但考虑到几乎不费什么成本（除了 Google One 订阅费），已经相当实用。&lt;/p&gt;
&lt;p&gt;&amp;lt;ImageWrapper
src={ImageRealSense.src}
alt=&quot;Shaded RealSense D435i model in Blender&quot;
caption=&quot;Blender 中带材质的 RealSense D435i 模型&quot;
class=&quot;w-full h-full&quot;
/&amp;gt;&lt;/p&gt;
&lt;p&gt;&amp;lt;ImageWrapper
src={ImageRealSense2.src}
alt=&quot;Shaded RealSense D435i model in Blender&quot;
caption=&quot;Blender 中带材质的 RealSense D435i 模型&quot;
class=&quot;w-full h-full&quot;
/&amp;gt;&lt;/p&gt;
&lt;hr /&gt;
&lt;p&gt;作者：欧亚非；语法与概念准确性由 ChatGPT 审校。原文以英文撰写，并在 ChatGPT 的协助下翻译；最终审阅与润色由欧亚非完成。&lt;br /&gt;
© 2026 欧亚非。保留所有权利。&lt;/p&gt;
</content:encoded></item><item><title>CRESSim–MPM: A Material Point Method Library for Surgical Soft Body Simulation with Cutting and Suturing</title><link>https://www.yafei.dev/posts/pub-ou2025cressim/</link><guid isPermaLink="true">https://www.yafei.dev/posts/pub-ou2025cressim/</guid><description>By Yafei Ou, Mahdi Tavakoli. 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2025.</description><pubDate>Sun, 19 Oct 2025 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;import YouTubeVideo from &apos;../../../components/misc/YouTubeVideo.astro&apos;;&lt;/p&gt;
&lt;h2&gt;Full Article&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;CRESSim-MPM: A Material Point Method Library for Surgical Soft Body Simulation with Cutting and Suturing&lt;/strong&gt;&lt;br /&gt;
Yafei Ou, Mahdi Tavakoli&lt;br /&gt;
&lt;em&gt;2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;IEEE Xplore: &lt;a href=&quot;https://ieeexplore.ieee.org/document/11247388&quot;&gt;https://ieeexplore.ieee.org/document/11247388&lt;/a&gt;&lt;br /&gt;
arXiv: &lt;a href=&quot;https://arxiv.org/abs/2502.18437&quot;&gt;https://arxiv.org/abs/2502.18437&lt;/a&gt;&lt;/p&gt;
&lt;h2&gt;Abstract&lt;/h2&gt;
&lt;p&gt;A number of recent studies have focused on developing surgical simulation platforms to train machine learning (ML) agents or models with synthetic data for surgical assistance. While existing platforms excel at tasks such as rigid body manipulation and soft body deformation, they struggle to simulate more complex soft body behaviors like cutting and suturing. A key challenge lies in modeling soft body fracture and splitting using the finite-element method (FEM), which is the predominant approach in current platforms. Additionally, the two-way suture needle/thread contact inside a soft body is further complicated when using FEM. In this work, we use the material point method (MPM) for such challenging simulations and propose new rigid geometries and soft-rigid contact methods specifically designed for them. We introduce CRESSim-MPM, a GPU-accelerated MPM library that integrates multiple MPM solvers and incorporates surgical geometries for cutting and suturing, serving as a specialized physics engine for surgical applications. It is further integrated into Unity, requiring minimal modifications to existing projects for soft body simulation. We demonstrate the simulator&apos;s capabilities in real-time simulation of cutting and suturing on soft tissue and provide an initial performance evaluation of different MPM solvers when simulating varying numbers of particles.&lt;/p&gt;
&lt;p&gt;&amp;lt;YouTubeVideo videoId=&quot;IwEsBQ508YU&quot; /&amp;gt;&lt;/p&gt;
&lt;h2&gt;Code&lt;/h2&gt;
&lt;h3&gt;CUDA/C++ library&lt;/h3&gt;
&lt;p&gt;::github{repo=&quot;yafei-ou/cressim-mpm&quot;}&lt;/p&gt;
&lt;h2&gt;Citation&lt;/h2&gt;
&lt;pre&gt;&lt;code&gt;@article{ou2025cressim,
  title={CRESSim-MPM: A Material Point Method Library for Surgical Soft Body Simulation with Cutting and Suturing},
  author={Ou, Yafei and Tavakoli, Mahdi},
  journal={arXiv preprint arXiv:2502.18437},
  year={2025}
}
&lt;/code&gt;&lt;/pre&gt;
</content:encoded></item><item><title>Learning Autonomous Surgical Irrigation and Suction with the da Vinci Research Kit Using Reinforcement Learning</title><link>https://www.yafei.dev/posts/pub-ou2024learning/</link><guid isPermaLink="true">https://www.yafei.dev/posts/pub-ou2024learning/</guid><description>By Yafei Ou, Mahdi Tavakoli. IEEE Transactions on Automation Science and Engineering, 2025.</description><pubDate>Wed, 04 Jun 2025 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;import YouTubeVideo from &apos;../../../components/misc/YouTubeVideo.astro&apos;;&lt;/p&gt;
&lt;h2&gt;Full Article&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;Learning Autonomous Surgical Irrigation and Suction with the da Vinci Research Kit Using Reinforcement Learning&lt;/strong&gt;&lt;br /&gt;
Yafei Ou, Mahdi Tavakoli&lt;br /&gt;
&lt;em&gt;IEEE Transactions on Automation Science and Engineering&lt;/em&gt;, vol. 22, pp. 16753-16767, 2025&lt;/p&gt;
&lt;p&gt;IEEE Xplore: &lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/11023631&quot;&gt;https://ieeexplore.ieee.org/abstract/document/11023631&lt;/a&gt;&lt;br /&gt;
arXiv: &lt;a href=&quot;https://arxiv.org/abs/2411.14622&quot;&gt;https://arxiv.org/abs/2411.14622&lt;/a&gt;&lt;/p&gt;
&lt;h2&gt;Abstract&lt;/h2&gt;
&lt;p&gt;The irrigation-suction process is a common procedure to rinse and clean up the surgical field in minimally invasive surgery (MIS). In this process, surgeons first irrigate liquid, typically saline, into the surgical scene for rinsing and diluting the contaminant, and then suction the liquid out of the surgical field. While recent advances have shown promising results in the application of reinforcement learning (RL) for automating surgical subtasks, fewer studies have explored the automation of fluid-related tasks. In this work, we explore the automation of both steps in the irrigation-suction procedure and train two vision-based RL agents to complete irrigation and suction autonomously. To achieve this, a platform is developed for creating simulated surgical robot learning environments and for training agents, and two simulated learning environments are built for irrigation and suction with visually plausible fluid rendering capabilities. With techniques such as domain randomization (DR) and carefully designed reward functions, two agents are trained in the simulator and transferred to the real world. Individual evaluations of both agents show satisfactory real-world results. With an initial amount of around 5 grams of contaminants, the irrigation agent ultimately achieved an average of 2.21 grams remaining after a manual suction. As a comparison, fully manual operation by a human results in 1.90 grams remaining. The suction agent achieved 2.64 and 2.24 grams of liquid remaining across two trial groups with more than 20 and 30 grams of initial liquid in the container. Fully autonomous irrigation-suction trials reduce the contaminant in the container from around 5 grams to an average of 2.42 grams, although yielding a higher total weight remaining (4.40) due to residual liquid not suctioned. Further information about the project is available at &lt;a href=&quot;https://tbs-ualberta.github.io/CRESSim/&quot;&gt;this https URL&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;&amp;lt;YouTubeVideo videoId=&quot;yvgTuXm1IwQ&quot; /&amp;gt;&lt;/p&gt;
&lt;h2&gt;Code&lt;/h2&gt;
&lt;h3&gt;CRESSim (Unity Project)&lt;/h3&gt;
&lt;p&gt;::github{repo=&quot;tbs-ualberta/cressim&quot;}&lt;/p&gt;
&lt;h3&gt;PhysX 5 for Unity&lt;/h3&gt;
&lt;p&gt;::github{repo=&quot;yafei-ou/physx5-for-unity&quot;}&lt;/p&gt;
&lt;h3&gt;PhysX 5 for Unity Native Plugin (C++)&lt;/h3&gt;
&lt;p&gt;::github{repo=&quot;yafei-ou/physx5-native-plugin&quot;}&lt;/p&gt;
&lt;h2&gt;Citation&lt;/h2&gt;
&lt;pre&gt;&lt;code&gt;@article{ou2025learning,
  title={Learning autonomous surgical irrigation and suction with the da vinci research kit using reinforcement learning},
  author={Ou, Yafei and Tavakoli, Mahdi},
  journal={IEEE Transactions on Automation Science and Engineering},
  year={2025},
  volume={22},
  pages={16753-16767},
  publisher={IEEE}
}
&lt;/code&gt;&lt;/pre&gt;
</content:encoded></item><item><title>Thoughts on Modern C++</title><link>https://www.yafei.dev/posts/2025-05-17-thoughts-cpp/</link><guid isPermaLink="true">https://www.yafei.dev/posts/2025-05-17-thoughts-cpp/</guid><description>Some thoughts on C++ based on my own experience.</description><pubDate>Sat, 17 May 2025 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;Modern C++ is undeniably powerful.
But, as with any programming language, it is also often hated by many.
And there are certainly reasons.&lt;/p&gt;
&lt;h2&gt;The Legacy&lt;/h2&gt;
&lt;p&gt;I have been using C++ for around a year for some projects, but without formal training or specific tutorials.
I really started learning it by reading Nvidia&apos;s PhysX codebase, which I found very well-organized and performance-driven.
PhysX has been used by many games--as well as game engines like Unity.
On the other hand, the codebase is far from being &quot;modern&quot; from many C++ users&apos; view.
STL is not even used--no smart pointers, no &lt;code&gt;std::vector&lt;/code&gt;, all implemented within PhysX in an old fashioned way.&lt;/p&gt;
&lt;p&gt;It is not difficult to make some guesses as to the reasons:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;PhysX was initially developed around 20 years ago when the standard hadn&apos;t even introduced the modern features;&lt;/li&gt;
&lt;li&gt;PhysX really wants to squeeze that last bit of performance, especially on low-end devices;&lt;/li&gt;
&lt;li&gt;PhysX should support those devices that your regular program won&apos;t run on, &lt;em&gt;e.g.&lt;/em&gt; Nintendo Switch (NS). I don&apos;t even know how much of the standard Nintendo has implemented for the NS, as it is completely proprietary.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;I am not sure how each of these points has contributed to this design, and there are likely other factors influencing PhysX’s architecture.
But here&apos;s the practical question: &lt;strong&gt;should we always use modern C++ features when starting a new project now?&lt;/strong&gt;
I do not have the answer myself, but since I learned C++ by reading PhysX&apos;s codebase, I have stayed away from many modern features.&lt;/p&gt;
&lt;h2&gt;The Confusing Standard&lt;/h2&gt;
&lt;p&gt;I do like many of the modern additions and changes to the C++ standard, which simplify and unify many aspects.
Still, the standard’s wording sometimes leaves even experienced users confused.
I once blamed my own skill level until I read &lt;a href=&quot;https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2025/p1839r7.html&quot;&gt;P1839R7&lt;/a&gt; and realized ambiguities (and possibly defects) in the standard are a common headache.
Below is quoted from the paper:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;void print_hex(int n) {
    unsigned char* a = (unsigned char*)(&amp;amp;n);
    for (int i = 0; i &amp;lt; sizeof(int); ++i)
        printf(&quot;%02x &quot;, a[i]);
}

int main() {
    print_hex(123456);
}
&lt;/code&gt;&lt;/pre&gt;
&lt;blockquote&gt;
&lt;p&gt;In C, this is a valid program. On a little-endian machine where &lt;code&gt;sizeof(int) == 4&lt;/code&gt;, this will print &lt;code&gt;40 e2 01 00&lt;/code&gt;. In C++, this is widely assumed to be valid as well, and this functionality is widely used in existing code bases (think of binary file formats, hex viewers, and many other low-level use cases).&lt;/p&gt;
&lt;p&gt;However, surprisingly, in C++ this code has undefined behaviour under the current specification. In fact, it is impossible in C++ to directly access the object representation of an object (i.e. to read its underlying bytes), even for built-in types such as int. Instead, we would have to use memcpy to copy the bytes into a separate array of &lt;code&gt;unsigned char&lt;/code&gt;, and access them from there. However, this workaround only works for trivially copyable types. It also directly violates one of the fundamental principles of C++: to leave no room for a lower-level language.&lt;/p&gt;
&lt;p&gt;The goal of this paper is to provide the necessary wording fixes to make accessing object representations such as in the code above defined behaviour. Existing compilers already assume that this should be valid. The goal of the paper is therefore to not require any changes to existing compilers or existing code, but to legalise existing code that already works in practice and was always intended to be valid.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Practically, compiler implementations and their documentation are the de facto standard, rather than the formal C++ standard.
One example is &lt;a href=&quot;https://stackoverflow.com/questions/11373203/accessing-inactive-union-member-and-undefined-behavior&quot;&gt;&lt;code&gt;union&lt;/code&gt; type punning&lt;/a&gt;.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;
#include &amp;lt;cstdint&amp;gt;

union Pun {
    std::uint32_t i;
    float         f;
};

int main() {
    Pun u;
    u.i = 0x3F800000;      // Bit pattern for 1.0f in IEEE-754
    std::cout &amp;lt;&amp;lt; u.f      // Reads the float interpretation
              &amp;lt;&amp;lt; std::endl;
    return 0;
}

&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The use case is well-defined by the C standard, but is UB in C++.
The C++ standard does not define the behavior (likely deliberately) and leaves more freedom for compilers&apos; implementation.
Practically I don&apos;t see any compiler that does not support this.
But it is very confusing to look up and be certain.
GCC states this explicitly:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;The practice of reading from a different union member than the one most recently written to (called “type-punning”) is common. Even with &lt;code&gt;-fstrict-aliasing&lt;/code&gt;, type-punning is allowed, provided the memory is accessed through the union type. So, the code above works as expected. See Structures, Unions, Enumerations, and Bit-Fields.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;However, I can&apos;t find any information about MSVC&apos;s treatment on this case, although it is obviously supported.&lt;/p&gt;
&lt;p&gt;The ambiguity on the wording of the standard, either by design or by oversight, makes it almost a common practice to wait for at least 5 years before a standard can be used until things stabilize.
There seems to be no hope for improvements in the situation.&lt;/p&gt;
&lt;h2&gt;How About CUDA?&lt;/h2&gt;
&lt;p&gt;CUDA&apos;s rise for GPGPU, and especially with recent hype in machine learning, has attracted another unique group of C++ users.
NVCC&apos;s implementation of C++ is very unique, as the code is compiled for the GPU device instead of the CPU.
This adds an additional layer of confusion.&lt;/p&gt;
&lt;p&gt;One example is pointer casting.
Using this for type punning is UB that violates the strict aliasing rule (although at least MSVC shouldn&apos;t have trouble with this).
It is usually more proper to use &lt;code&gt;union&lt;/code&gt; or &lt;code&gt;memcpy&lt;/code&gt; to achieve this (although &lt;code&gt;union&lt;/code&gt; doesn&apos;t seem portable as per the standard), but Nvidia&apos;s &lt;a href=&quot;https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#atomic-functions&quot;&gt;official documentation&lt;/a&gt; implies that at least this will definitely work for their GPUs.
Below is taken from the documentation:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;#if __CUDA_ARCH__ &amp;lt; 600
__device__ double atomicAdd(double* address, double val)
{
    unsigned long long int* address_as_ull =
                              (unsigned long long int*)address;
    unsigned long long int old = *address_as_ull, assumed;

    do {
        assumed = old;
        old = atomicCAS(address_as_ull, assumed,
                        __double_as_longlong(val +
                               __longlong_as_double(assumed)));

    // Note: uses integer comparison to avoid hang in case of NaN (since NaN != NaN)
    } while (assumed != old);

    return __longlong_as_double(old);
}
#endif
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Casting from a &lt;code&gt;double*&lt;/code&gt; to a &lt;code&gt;unsigned long long int*&lt;/code&gt; and dereferencing it is obviously UB.
But this code snippet suggests that it should be well-defined, at least for device code.&lt;/p&gt;
&lt;h2&gt;Conclusion&lt;/h2&gt;
&lt;p&gt;My experience with developing a cross-platform C++/CUDA project has been painful, and the aspects mentioned above have been lingering in my mind since the beginning.&lt;/p&gt;
&lt;p&gt;There has not been a practical alternative to C++ at least for CUDA programming (or even broadly, most GPGPU programming).
I guess the only way is to live with it, live with a confusing standard, and live with UBs in my code.&lt;/p&gt;
&lt;hr /&gt;
&lt;p&gt;Written by Yafei Ou and reviewed for grammar and conceptual accuracy by ChatGPT (o4-mini-high).&lt;br /&gt;
© 2025 Yafei Ou. All rights reserved.&lt;/p&gt;
</content:encoded></item><item><title>Autonomous Soft-Tissue Needle Steering Using Reinforcement Learning Guided by Human Input</title><link>https://www.yafei.dev/posts/pub-ou2024autonomous-2/</link><guid isPermaLink="true">https://www.yafei.dev/posts/pub-ou2024autonomous-2/</guid><description>By Yafei Ou, Mahdi Tavakoli. Journal of Medical Robotics Research, 2024.</description><pubDate>Tue, 24 Dec 2024 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;import YouTubeVideo from &apos;../../../components/misc/YouTubeVideo.astro&apos;;&lt;/p&gt;
&lt;h2&gt;Full Article&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;Autonomous Soft-Tissue Needle Steering Using Reinforcement Learning Guided by Human Input&lt;/strong&gt;&lt;br /&gt;
Yafei Ou, Mahdi Tavakoli&lt;br /&gt;
&lt;em&gt;Journal of Medical Robotics Research&lt;/em&gt;, vol. 09, no. 03n04, 2024.&lt;/p&gt;
&lt;p&gt;Link: &lt;a href=&quot;https://www.worldscientific.com/doi/abs/10.1142/S2424905X24400105&quot;&gt;https://www.worldscientific.com/doi/abs/10.1142/S2424905X24400105&lt;/a&gt;&lt;/p&gt;
&lt;h2&gt;Abstract&lt;/h2&gt;
&lt;p&gt;Soft-tissue needle steering, where a deformable needle is inserted into the tissue to guide its tip to a desired position, is a common minimally invasive surgery (MIS) procedure. The diverse types of needles and complex tissue dynamics limit the use of existing approaches that utilize models of the needle and the tissue for automating the task. In this work, we employ a data-driven approach using deep reinforcement learning (DRL) to achieve autonomous needle steering by viewing it as a multi-goal reinforcement learning problem. Human interventions are incorporated during training to accelerate learning and reduce catastrophic failures. Generative adversarial imitation learning (GAIL) is combined with regular DRL by utilizing a hindsight relabeling scheme for human interventions to encourage the agent to imitate human behavior. To emulate the sim-to-real process, an agent is first trained in a simplistic simulation environment for needle steering and then transferred to a sophisticated one considered as the real world with fine-tuning (sim-to-sim). Experimental results show that with human interventions, the proposed method outperforms the other compared DRL approaches and can achieve good performance with only 2,000 training steps in the complex simulation environment, achieving an average return comparable to that of a 55,000-step agent trained from scratch.&lt;/p&gt;
&lt;h2&gt;Citation&lt;/h2&gt;
&lt;pre&gt;&lt;code&gt;@article{ou2024autonomous,
  title={Autonomous Soft-Tissue Needle Steering Using Reinforcement Learning Guided by Human Input},
  author={Ou, Yafei and Tavakoli, Mahdi},
  journal={Journal of Medical Robotics Research},
  volume={9},
  number={03n04},
  pages={2440010},
  year={2024},
  publisher={World Scientific}
}
&lt;/code&gt;&lt;/pre&gt;
</content:encoded></item><item><title>Autonomous Blood Suction for Robot-Assisted Surgery: A Sim-to-Real Reinforcement Learning Approach</title><link>https://www.yafei.dev/posts/pub-ou2024autonomous/</link><guid isPermaLink="true">https://www.yafei.dev/posts/pub-ou2024autonomous/</guid><description>By Yafei Ou, Abed Soleymani, Xingyu Li, Mahdi Tavakoli. IEEE Robotics and Automation Letters, 2024.</description><pubDate>Mon, 01 Jul 2024 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;import YouTubeVideo from &apos;../../../components/misc/YouTubeVideo.astro&apos;;&lt;/p&gt;
&lt;h2&gt;Full Article&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;Autonomous Blood Suction for Robot-Assisted Surgery: A Sim-to-Real Reinforcement Learning Approach&lt;/strong&gt;&lt;br /&gt;
Yafei Ou, Abed Soleymani, Xingyu Li, Mahdi Tavakoli&lt;br /&gt;
&lt;em&gt;IEEE Robotics and Automation Letters&lt;/em&gt;, vol. 9, no. 8, pp. 7246-7253, Aug. 2024&lt;/p&gt;
&lt;p&gt;IEEE Xplore: &lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/10578312&quot;&gt;https://ieeexplore.ieee.org/abstract/document/10578312&lt;/a&gt;&lt;/p&gt;
&lt;h2&gt;Abstract&lt;/h2&gt;
&lt;p&gt;Recent applications of deep reinforcement learning (DRL) in surgical autonomy have shown promising results in automating various surgical sub-tasks. While most of these studies consider the rigid and soft body dynamics in the surgery such as tissue deformation, only a few have investigated the situation where fluid is present. However, the presence of fluids, particularly blood, is common in surgeries, and interacting with them adds additional challenges to task automation. In this work, we investigate the use of DRL in automating blood suction, a common surgical sub-task where blood is removed from the surgical field. We build a blood suction simulation environment based on position-based fluids (PBF), train an agent with domain-randomized environment parameters through curriculum learning, and obtain a generalizable policy that can be applied to various shapes of tissue and types of liquid. Real-world experiments show that the agent can perform autonomous suction in different tissue models with different amounts and types of liquid, and only one of the 50 trials resulted in more than 3 ml of blood remaining.&lt;/p&gt;
&lt;p&gt;&amp;lt;YouTubeVideo videoId=&quot;u75v2EOT38Q&quot; /&amp;gt;&lt;/p&gt;
&lt;h2&gt;Citation&lt;/h2&gt;
&lt;pre&gt;&lt;code&gt;@article{ou2024autonomous,
  title={Autonomous Blood Suction for Robot-Assisted Surgery: A Sim-to-Real Reinforcement Learning Approach},
  author={Ou, Yafei and Soleymani, Abed and Li, Xingyu and Tavakoli, Mahdi},
  journal={IEEE Robotics and Automation Letters},
  year={2024},
  publisher={IEEE}
}
&lt;/code&gt;&lt;/pre&gt;
</content:encoded></item><item><title>A Realistic Surgical Simulator for Non-Rigid and Contact-Rich Manipulation in Surgeries with the da Vinci Research Kit</title><link>https://www.yafei.dev/posts/pub-ou2024realistic/</link><guid isPermaLink="true">https://www.yafei.dev/posts/pub-ou2024realistic/</guid><description>By Yafei Ou, Sadra Zargarzadeh, Paniz Sedighi, Mahdi Tavakoli. 21st International Conference on Ubiquitous Robots (UR), 2024</description><pubDate>Mon, 08 Apr 2024 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;import YouTubeVideo from &apos;../../../components/misc/YouTubeVideo.astro&apos;;&lt;/p&gt;
&lt;h2&gt;Full Article&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;A Realistic Surgical Simulator for Non-Rigid and Contact-Rich Manipulation in Surgeries with the da Vinci Research Kit&lt;/strong&gt;&lt;br /&gt;
Yafei Ou, Sadra Zargarzadeh, Paniz Sedighi, Mahdi Tavakoli&lt;br /&gt;
&lt;em&gt;2024 21st International Conference on Ubiquitous Robots (UR)&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;IEEE Xplore: &lt;a href=&quot;https://ieeexplore.ieee.org/document/10597513&quot;&gt;https://ieeexplore.ieee.org/document/10597513&lt;/a&gt;&lt;br /&gt;
arXiv: &lt;a href=&quot;https://arxiv.org/abs/2404.05888&quot;&gt;https://arxiv.org/abs/2404.05888&lt;/a&gt;&lt;/p&gt;
&lt;h2&gt;Abstract&lt;/h2&gt;
&lt;p&gt;Realistic real-time surgical simulators play an increasingly important role in surgical robotics research, such as surgical robot learning and automation, and surgical skills assessment. Although there are a number of existing surgical simulators for research, they generally lack the ability to simulate the diverse types of objects and contact-rich manipulation tasks typically present in surgeries, such as tissue cutting and blood suction. In this work, we introduce CRESSim, a realistic surgical simulator based on PhysX 5 for the da Vinci Research Kit (dVRK) that enables simulating various contact-rich surgical tasks involving different surgical instruments, soft tissue, and body fluids. The real-world dVRK console and the master tool manipulator (MTM) robots are incorporated into the system to allow for teleoperation through virtual reality (VR). To showcase the advantages and potentials of the simulator, we present three examples of surgical tasks, including tissue grasping and deformation, blood suction, and tissue cutting. These tasks are performed using the simulated surgical instruments, including the large needle driver, suction irrigator, and curved scissor, through VR-based teleoperation.&lt;/p&gt;
&lt;p&gt;&amp;lt;YouTubeVideo videoId=&quot;GIBHPmgqJcs&quot; /&amp;gt;&lt;/p&gt;
&lt;h2&gt;Code&lt;/h2&gt;
&lt;h3&gt;CRESSim (Unity Project)&lt;/h3&gt;
&lt;p&gt;::github{repo=&quot;tbs-ualberta/cressim&quot;}&lt;/p&gt;
&lt;h3&gt;PhysX 5 for Unity&lt;/h3&gt;
&lt;p&gt;::github{repo=&quot;yafei-ou/physx5-for-unity&quot;}&lt;/p&gt;
&lt;h3&gt;PhysX 5 for Unity Native Plugin (C++)&lt;/h3&gt;
&lt;p&gt;::github{repo=&quot;yafei-ou/physx5-native-plugin&quot;}&lt;/p&gt;
&lt;h3&gt;Citation&lt;/h3&gt;
&lt;pre&gt;&lt;code&gt;@inproceedings{ou2024realistic,
  title={A Realistic Surgical Simulator for Non-Rigid and Contact-Rich Manipulation in Surgeries with the da Vinci Research Kit}, 
  author={Ou, Yafei and Zargarzadeh, Sadra and Sedighi, Paniz and Tavakoli, Mahdi},
  booktitle={2024 21st International Conference on Ubiquitous Robots (UR)}, 
  year={2024},
  pages={64-70}
}
&lt;/code&gt;&lt;/pre&gt;
</content:encoded></item><item><title>Towards Safe and Efficient Reinforcement Learning for Surgical Robots Using Real-Time Human Supervision and Demonstration</title><link>https://www.yafei.dev/posts/pub-ou2023towards/</link><guid isPermaLink="true">https://www.yafei.dev/posts/pub-ou2023towards/</guid><description>By Yafei Ou, Mahdi Tavakoli. 2023 International Symposium on Medical Robotics (ISMR), 2023.</description><pubDate>Wed, 19 Apr 2023 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;import YouTubeVideo from &apos;../../../components/misc/YouTubeVideo.astro&apos;;&lt;/p&gt;
&lt;h2&gt;Full Article&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;Towards Safe and Efficient Reinforcement Learning for Surgical Robots Using Real-Time Human Supervision and Demonstration&lt;/strong&gt;&lt;br /&gt;
Yafei Ou, Mahdi Tavakoli&lt;br /&gt;
&lt;em&gt;2023 International Symposium on Medical Robotics (ISMR)&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;IEEE Xplore: &lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/10130214&quot;&gt;https://ieeexplore.ieee.org/abstract/document/10130214&lt;/a&gt;&lt;/p&gt;
&lt;h2&gt;Abstract&lt;/h2&gt;
&lt;p&gt;Recent research in surgical robotics has focused on increasing the level of autonomy in order to reduce the workload of surgeons. While deep reinforcement learning (DRL) has shown promising results in automating some surgical subtasks, due to its demand for a large number of random explorations, safety and learning efficiency remain the primary challenges when applying it to surgical robot learning. In this work, we present a DRL framework with real-time human supervision during the training process for surgical robot learning to avoid significant failures and speed up training. A novel training methodology based on the combination of DRL and generative adversarial imitation learning (GAIL) is proposed to further improve learning efficiency by imitating human behaviors. The proposed method is validated using two simulated environments, where human intervention is performed via teleoperation. Results show that our method outperforms baseline algorithms and can achieve safe and efficient learning.&lt;/p&gt;
&lt;h2&gt;Citation&lt;/h2&gt;
&lt;pre&gt;&lt;code&gt;@inproceedings{ou2023towards,
  title={Towards safe and efficient reinforcement learning for surgical robots using real-time human supervision and demonstration},
  author={Ou, Yafei and Tavakoli, Mahdi},
  booktitle={2023 International Symposium on Medical Robotics (ISMR)},
  pages={1--7},
  year={2023},
  organization={IEEE}
}
&lt;/code&gt;&lt;/pre&gt;
</content:encoded></item><item><title>Robot Learning Incorporating Human Interventions in the Real World for Autonomous Surgical Endoscopic Camera Control</title><link>https://www.yafei.dev/posts/pub-ou2023robot/</link><guid isPermaLink="true">https://www.yafei.dev/posts/pub-ou2023robot/</guid><description>By Yafei Ou, Mahdi Tavakoli. Journal of Medical Robotics Research, 2023.</description><pubDate>Fri, 10 Mar 2023 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;import YouTubeVideo from &apos;../../../components/misc/YouTubeVideo.astro&apos;;&lt;/p&gt;
&lt;h2&gt;Full Article&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;Robot Learning Incorporating Human Interventions in the Real World for Autonomous Surgical Endoscopic Camera Control&lt;/strong&gt;&lt;br /&gt;
Yafei Ou, Mahdi Tavakoli&lt;br /&gt;
&lt;em&gt;Journal of Medical Robotics Research&lt;/em&gt;, vol. 08, no. 03n04, p. 2340004, 2023.&lt;/p&gt;
&lt;p&gt;Link: &lt;a href=&quot;https://www.worldscientific.com/doi/abs/10.1142/S2424905X23400044&quot;&gt;https://www.worldscientific.com/doi/abs/10.1142/S2424905X23400044&lt;/a&gt;&lt;/p&gt;
&lt;h2&gt;Abstract&lt;/h2&gt;
&lt;p&gt;Recent studies in surgical robotics have focused on automating common surgical subtasks such as grasping and manipulation using deep reinforcement learning (DRL). In this work, we consider surgical endoscopic camera control for object tracking e.g. using the endoscopic camera manipulator (ECM) from the da Vinci Research Kit (dVRK) (Intuitive Inc., Sunnyvale, CA, USA) as a typical surgical robot learning task. A DRL policy for controlling the robot joint space movements is first trained in a simulation environment and then continues the learning in the real world. To speed up training and avoid significant failures (in this case, losing view of the object), human interventions are incorporated into the training process and regular DRL is combined with generative adversarial imitation learning (GAIL) to encourage imitating human behaviors. Experiments show that an average reward of 159.8 can be achieved within 1000 steps compared to only 121.8 without human interventions, and the view of the moving object is lost only twice during the training process out of 3 trials. These results show that human interventions can improve learning speed and significantly reduce failures during the training process.&lt;/p&gt;
&lt;h2&gt;Citation&lt;/h2&gt;
&lt;pre&gt;&lt;code&gt;@article{ou2023robot,
  title={Robot learning incorporating human interventions in the real world for autonomous surgical endoscopic camera control},
  author={Ou, Yafei and Zargarzadeh, Sadra and Tavakoli, Mahdi},
  journal={Journal of Medical Robotics Research},
  volume={8},
  number={03n04},
  pages={2340004},
  year={2023},
  publisher={World Scientific}
}
&lt;/code&gt;&lt;/pre&gt;
</content:encoded></item><item><title>Sim-to-Real Surgical Robot Learning and Autonomous Planning for Internal Tissue Points Manipulation Using Reinforcement Learning</title><link>https://www.yafei.dev/posts/pub-ou2023sim/</link><guid isPermaLink="true">https://www.yafei.dev/posts/pub-ou2023sim/</guid><description>By Yafei Ou, Mahdi Tavakoli. IEEE Robotics and Automation Letters, 2023.</description><pubDate>Fri, 10 Mar 2023 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;import YouTubeVideo from &apos;../../../components/misc/YouTubeVideo.astro&apos;;&lt;/p&gt;
&lt;h2&gt;Full Article&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;Sim-to-Real Surgical Robot Learning and Autonomous Planning for Internal Tissue Points Manipulation Using Reinforcement Learning&lt;/strong&gt;&lt;br /&gt;
Yafei Ou, Mahdi Tavakoli&lt;br /&gt;
&lt;em&gt;IEEE Robotics and Automation Letters&lt;/em&gt;, vol. 8, no. 5, pp. 2502-2509, May 2023&lt;/p&gt;
&lt;p&gt;IEEE Xplore: &lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/10065553&quot;&gt;https://ieeexplore.ieee.org/abstract/document/10065553&lt;/a&gt;&lt;/p&gt;
&lt;h2&gt;Abstract&lt;/h2&gt;
&lt;p&gt;Indirect simultaneous positioning (ISP), where internal tissue points are placed at desired locations indirectly through the manipulation of boundary points, is a type of subtask frequently performed in robotic surgeries. Although challenging due to complex tissue dynamics, automating the task can potentially reduce the workload of surgeons. This letter presents a sim-to-real framework for learning to automate the task without interacting with a real environment, and for planning preoperatively to find the grasping points that minimize local tissue deformation. A control policy is learned using deep reinforcement learning (DRL) in the FEM-based simulation environment and transferred to real-world situation. Grasping points are planned in the simulator by utilizing the trained policy using Bayesian optimization (BO). Inconsistent simulation performance is overcome by formulating the problem as a state augmented Markov decision process (MDP). Experimental results show that the learned policy places the internal tissue points accurately, and that the planned grasping points yield small tissue deformation among the trials. The proposed learning and planning scheme is able to automate internal tissue point manipulation in surgeries and has the potential to be generalized to complex surgical scenarios.&lt;/p&gt;
&lt;p&gt;&amp;lt;YouTubeVideo videoId=&quot;PKHARy8CbuQ&quot; /&amp;gt;&lt;/p&gt;
&lt;h2&gt;Citation&lt;/h2&gt;
&lt;pre&gt;&lt;code&gt;@article{ou2024autonomous,
  title={Autonomous Soft-Tissue Needle Steering Using Reinforcement Learning Guided by Human Input},
  author={Ou, Yafei and Tavakoli, Mahdi},
  journal={Journal of Medical Robotics Research},
  volume={9},
  number={03n04},
  pages={2440010},
  year={2024},
  publisher={World Scientific}
}
&lt;/code&gt;&lt;/pre&gt;
</content:encoded></item><item><title>Site Changelog</title><link>https://www.yafei.dev/posts/site-changelog/</link><guid isPermaLink="true">https://www.yafei.dev/posts/site-changelog/</guid><description>Changelog of this site.</description><pubDate>Sat, 01 Jan 2022 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;The changelog of the site.
The published date is set to be old enough so that this post will always appear at the bottom.&lt;/p&gt;
&lt;h2&gt;2026-01-22&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Added astrojs/mdx and use MDX for most of the posts&lt;/li&gt;
&lt;li&gt;Added some other publications&lt;/li&gt;
&lt;li&gt;Increased cover image height&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;2025-05-18&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Fixed an issue due to some characters in path (in tags)&lt;/li&gt;
&lt;li&gt;Added the 404 page from Fuwari&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;2025-05-17&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Migrated to Fuwari&lt;/li&gt;
&lt;li&gt;Added publications and CV pages&lt;/li&gt;
&lt;li&gt;Added posts on publications&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;2024-09-07&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Updated the CV page and the PDF version.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;2024-08-31&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Updated the CV page and the PDF version.&lt;/li&gt;
&lt;li&gt;Changed the website title to &quot;Yafei&apos;s Archive&quot;&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;2024-07-28&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Added CV page&lt;/li&gt;
&lt;li&gt;Removed redundant pages&lt;/li&gt;
&lt;li&gt;Added Publications page (only one paper has been added)&lt;/li&gt;
&lt;/ul&gt;
</content:encoded></item></channel></rss>