Python node2vec(Gensim Word2Vec)“进程以退出代码134完成

Python node2vec (Gensim Word2Vec) quot;Process finished with exit code 134 (interrupted by signal 6: SIGABRT)quot;(Python node2vec(Gensim Word2Vec)“进程以退出代码134完成(被信号6中断:SIGABRT))

问题描述

我正在用 Python 开发 node2vec，它在内部使用 Gensim 的 Word2Vec.

当我使用小型数据集时，代码运行良好.但是一旦我尝试在大型数据集上运行相同的代码，代码就会崩溃.

错误:进程以退出代码 134 结束(被信号 6:SIGABRT 中断).

给出错误的行是

model = Word2Vec(walks, size=args.dimensions,窗口=args.window_size，min_count=0，sg=1，工人=args.workers，iter=args.iter)

我正在使用 pycharm 和 python 3.5.

知道发生了什么吗?我找不到任何可以解决我的问题的帖子.

您几乎可以肯定内存不足 - 这会导致操作系统使用 SIGABRT 中止您的内存使用进程.p>

一般来说，解决这个问题意味着查看您的代码是如何使用内存的，以及在发生故障之前和发生故障时的情况.(然而，过多的大容量内存使用的实际泄漏"可能会更早 - 只有最后一个小/适当的增量会触发错误.)

特别是使用 Python，以及利用 Gensim Word2Vec 类的 node2vec 工具，可以尝试的一些事情包括:

在尝试期间观察 Python 进程大小的读数.

至少启用 Python 日志记录到 INFO 级别，以了解有关导致崩溃的更多信息.

此外，请务必:

1. 优化你的 walks 可迭代不组成一个大的内存列表.(Gensim 的 Word2Vec 可以处理 任何 长度的语料库，不包括那些远大于 RAM 的语料库，只要 (a) 语料库通过可重复迭代从磁盘流式传输Python 序列；和 (b) 模型的 unique 单词/节点标记数可以在 RAM 中建模.)
2. 确保模型中唯一词(令牌/节点)的数量不需要大于 RAM 允许的模型.日志输出一旦启用，将在主模型分配(可能失败)发生之前显示所涉及的原始大小.(如果失败，可以:(a) 使用具有更多 RAM 的系统来容纳您的全部节点；或 (b) 或使用更高的 min_count 值来丢弃更多不太重要的节点.)

如果您的进程以退出代码 134 结束(被信号 6:SIGABRT 中断) 错误不涉及 Python、Gensim 和 &Word2Vec，您应该:

1. 结合您的触发情况的更具体细节搜索该错误的发生情况 - 导致您的错误的工具/库和代码行.
2. 查看适合您情况的通用内存分析工具，以确定您的代码在哪里(甚至在最终错误之前很久)可能消耗了几乎所有可用的 RAM.

I am working on node2vec in Python, which uses Gensim's Word2Vec internally.

When I am using small dataset the code works well. But as soon as I try to run the same code on large dataset, the code crashes.

Error: Process finished with exit code 134 (interrupted by signal 6: SIGABRT).

The line which is giving error is

model = Word2Vec(walks, size=args.dimensions,
                 window=args.window_size, min_count=0, sg=1,
                 workers=args.workers, iter=args.iter)

I am using pycharm and python 3.5.

Any idea what is happening? I could not found any post which could solve my problem.

You are almost certainly running out of memory – which causes the OS to abort your memory-using process with the SIGABRT.

In general, solving this means looking at how your code is using memory, leading up to and at the moment of failure. (The actual 'leak' of excessive bulk memory usage might, however, be arbitrarily earlier - with only the last small/proper increment triggering the error.)

Specifically with the usage of Python, and the node2vec tool which makes use of the Gensim Word2Vec class, some things to try include:

Watch a readout of the Python process size during your attempts.

Enable Python logging to at least the INFO level to see more about what's happening leading-up to the crash.

Further, be sure to:

1. Optimize your walks iterable to not compose a large in-memory list. (Gensim's Word2Vec can work on a corpus of any length, iuncluding those far larger than RAM, as long as (a) the corpus is streamed from disk via a re-iterable Python sequence; and (b) the model's number of unique word/node tokens can be modeled within RAM.)
2. Ensure the number of unique words (tokens/nodes) in your model doesn't require a model larger than RAM allows. Logging output, once enabled, will show the raw sizes involved just before the main model-allocation (which is likely failing) happens. (If it fails, either: (a) use a system with more RAM to accomdate your full set of nodes; or (b) or use a higher min_count value to discard more less-important nodes.)

If your Process finished with exit code 134 (interrupted by signal 6: SIGABRT) error does not involve Python, Gensim, & Word2Vec, you should instead:

1. Search for occurrences of that error combined with more specific details of your triggering situations - the tools/libraries and lines-of-code that create your error.
2. Look into general memory-profiling tools for your situation, to identify where (even long before the final error) your code might be consuming almost-all of the available RAM.

这篇关于Python node2vec(Gensim Word2Vec)“进程以退出代码134完成(被信号6中断:SIGABRT)"的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持编程学习网！