Java app通过JNI调用C++ DLL；如何最好地分配内存?

Java app calls C++ DLL via JNI; how best to allocate memory?(Java app通过JNI调用C++ DLL；如何最好地分配内存?)

问题描述

问题的基本摘要是:如何最好地优化我的内存分配，为我通过 JNI 访问的 DLL 提供尽可能多的内存?我应该以最小化什么为目标，我应该以什么为最大目标，等等.

Basic summary of question is: How do I best optimize my memory allocation to give as much memory to the DLLs I access through JNI as possible? What should I aim to minimize, what should I aim to maximize, etc.

系统:在具有 4 GB RAM 的 32 位系统中将 JBoss 6 作为 Windows 32 服务运行.我确实了解 Java Heap 的内存有最大限制.JVM 是 JRE1.6.0_26

SYSTEM: Running JBoss 6 as a Windows 32 Service in a 32-bit system with 4 GB RAM. I do understand there are maximum restrictions on memory for Java Heap. JVM is JRE1.6.0_26

服务:JBoss 下安装了一个 webapp，它接收来自客户端的请求；每个请求都通过 JNI 调用 C++ 构建的 DLL 以某种方式处理图像文件.

SERVICE: Installed under JBoss is a webapp which receives requests from clients; each request calls the C++-built DLL through JNI to process an image file in some fashion or other.

问题:有时，对于较大的或部分(不是全部)LZW 压缩图像，调用 java 类会收到一条消息，指出 DLL 经历了全局内存耗尽并且未能完成请求的进程.

ISSUE: Occasionally, with larger or some (not all) LZW-compression images, the calling java class receives a message that the DLL experienced a Global Memory Depletion and failed to complete the requested process.

除了基本的 Windows 进程之外，服务器上没有任何其他活动在运行.

There is nothing else actively running on the server beyond basic windows processes.

目前JBOSS App Server内存设置如下，但可能超标:

Current JBOSS App Server memory settings are as follows, but may be excessive:

-Xms1024m -Xmx1024m -Xss1024k -XX:MaxPermSize=128m

-Xms1024m -Xmx1024m -Xss1024k -XX:MaxPermSize=128m

我正在尝试确定最佳内存设置，以便为 JNI DLL 提供尽可能多的资源，据我所知，JNI 不使用分配给 Java 堆的任何内存.

I am trying to determine the best memory settings to give as much resources to the JNI DLL, as I understand JNI does not use any memory allocated to the Java Heap.

我已阅读这些内容，但发现它们对回答我的问题没有帮助:

I have read these, but did not find them helpful to answer my question:

Java JNI:内存分配/分区

jconsole可以用来识别 JNI C++ 对象中的内存泄漏?

目前提供的两个答案并未解决固有问题.

一周后JBoss服务器的当前内存，JVM参数设置如上(TaskManager指示java.exe进程为750,672k)

Total Memory Pools: 5

Pool: Code Cache (Non-heap memory)

    Peak Usage : init:2359296, used:7317312, committed:7438336, max:50331648
    Current Usage : init:2359296, used:7306496, committed:7438336, max:50331648


        |---------| committed:7.09Mb
        +---------------------------------------------------------------------+
        |/////////| | max:48Mb
        +---------------------------------------------------------------------+
        |---------| used:6.97Mb


Pool: PS Eden Space (Heap memory)

    Peak Usage : init:268500992, used:354811904, committed:354811904, max:355991552
    Current Usage : init:268500992, used:270153472, committed:354091008, max:354156544


        |--------------------------------------------------------------------| committed:337.69Mb
        +---------------------------------------------------------------------+
        |///////////////////////////////////////////////////// || max:337.75Mb
        +---------------------------------------------------------------------+
        |----------------------------------------------------| used:257.64Mb


Pool: PS Survivor Space (Heap memory)

    Peak Usage : init:44695552, used:44694896, committed:78643200, max:78643200
    Current Usage : init:44695552, used:0, committed:1835008, max:1835008


        |---------------------------------------------------------------------| committed:1.75Mb
        +---------------------------------------------------------------------+
        | | max:1.75Mb
        +---------------------------------------------------------------------+
        | used:0b


Pool: PS Old Gen (Heap memory)

    Peak Usage : init:715849728, used:123671968, committed:715849728, max:715849728
    Current Usage : init:715849728, used:104048648, committed:715849728, max:715849728


        |---------------------------------------------------------------------| committed:682.69Mb
        +---------------------------------------------------------------------+
        |////////// | max:682.69Mb
        +---------------------------------------------------------------------+
        |---------| used:99.23Mb


Pool: PS Perm Gen (Non-heap memory)

    Peak Usage : init:16777216, used:91989664, committed:134217728, max:134217728
    Current Usage : init:16777216, used:90956472, committed:90963968, max:134217728


        |----------------------------------------------| committed:86.75Mb
        +---------------------------------------------------------------------+
        |//////////////////////////////////////////////| | max:128Mb
        +---------------------------------------------------------------------+
        |----------------------------------------------| used:86.74Mb

推荐答案

JNI封装的native代码分配的内存分配给JVM进程，但不受你的Java代码控制.它不是堆的一部分，也不能通过 JVM 参数进行调整.基本上，使用本机 malloc 分配的任何内容都必须由该本机代码管理.如果您可以控制正在使用的库，则必须仔细检查并检查资源泄漏.如果将其用于长期存在的过程中，这一点尤其重要.

Memory allocated by the native code wrapped by JNI is allocated to the JVM process, but is not under the control of your Java code. It is not part of the heap, and is not tunable via JVM parameters. Basically, anything allocated with a native malloc must be managed by that native code. If you are in control of the libraries you are using, its imperative that you go through it and check for resource leaks. This is especially important if this is being used in a long lived process.

根据我的经验，最好的方法是通过提取 JVM 公开的 JMX 统计信息来检查您的实际内存使用情况.一旦你知道你的 Java 应用程序消耗了多少内存，你就会更好地了解在哪里设置你的最大堆设置.Permgen 空间用于类定义等，因此除非您正在执行大量动态类加载，否则您确实不需要太多内存.

In my experience the best approach would be to examine your actual memory use by pulling the JMX stats exposed by the JVM. Once you have an idea about how much memory your Java app consumes You'll have a better idea about where to set your max heap settings. Permgen space is used for class definitions and such, so you really shouldn't need much memory there unless you are doing a bunch of dynamic class loading.

虽然您无法调整 JNI 库的可用内存，但调整为堆保留的内存等可能会释放资源供库使用.

While you cannot tune the memory available for the JNI library, tuning the memory reserved for your heap and such will potentially free up resources for use by the library.

正如所料，将堆内存峰值加在一起大约为 1022.19(堆的最大大小).当堆耗尽时，将启动完整的 GC 运行并回收脏堆.根据您提供的数字，我建议从 Xmx512m 开始.这将为您的 JNI 代码提供喘息的空间.

As would be expected, adding the heap memory peaks together it comes out to about 1022.19 (the max size of your heap). When the heap is exhausted a full GC run is kicked off and dirty heap is reclaimed. Based on the numbers that you have provided, I'd suggest starting with a Xmx512m. This will give your JNI code room to breath.

如果您发现 JVM 由于过多的垃圾收集而崩溃，这意味着您很快就用完了 Java 堆，您可以增加该分配.但是，如果它以足够快的速度消耗 512mb 以引起明显的性能影响，那么任何没有显着增加的东西都不太可能产生很大的影响.这在很大程度上取决于您的程序、它消耗 Java 堆的速度以及完整 GC 运行的效率.

If you find that the JVM is thrashing due to excessive garbage collection, meaning that you're running out of Java heap too quickly, you could grow that allocation. However, if it is eating up 512mb rapidly enough to cause a noticeable performance impact, its unlikely that anything short of a significant increase will have much effect. This all depends heavily on your program, how quickly it eats the Java heap, and how effective the full GC run is.

这篇关于Java app通过JNI调用C++ DLL；如何最好地分配内存?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持编程学习网！