Habra's karmic curse

Unforeseen Consequences

“The Habr karma system and its impact on users” is a topic for a course at least

Theme about karma on "Picabu"

I could start this article with the fact that I have been reading Habr for a long time, but this will not be a completely accurate statement. The correct thesis would be: “I have been reading articles from Habr for a long time” - but I was not interested in what was happening inside the community when I decided to finally register this spring. This is a typical mistake of a person who comes to Habr from a search engine to read useful articles about the intricacies of programming or interesting news from the world of technology. As long as you see the portal only from this, positive side, you do not ask questions about what is happening under the hood. Of course, in comments or articles mentions of karma slipped from time to time - but after all, karma is on almost all major portals (I naively thought), this is normal for self-regulating Internet communities.

I had to seriously think about it after I suddenly lost the ability to write more than one comment in five minutes.

At the same time, outwardly everything went perfectly: my comments all the time pounded, my rating grew - and suddenly it turned out that I had negative karma. My whole long experience of Internet communication, all user habits, and commonplace common sense shouted to me that this was some kind of mistake: the approval indicator of a site user by other site users cannot simultaneously rise and fall! But I decided not to chop off the shoulder, but to conduct a little research, both analytical (in the form of studying the opinions of users about karma) and statistical (in the form of analysis of account indicators).

The history of user war with karma turned out to be very rich. With varying success, it has been going on for more than a dozen years, on its account dozens of blocked victims and several deleted articles. And, strangely enough, my problem (discrepancy between grades and karma) is practically not used in argumentation - even during the open API, these calculations were not used. Closer to all, only one commentator was selected in a relatively recent entry:

“Actually, what is interesting to find: are there people karmic-minded with big pluses for their comments?”

https://habr.com/en/company/habr/blog/437072/#comment_19650144

In the statistical part, you can see that yes, there are such people. But even without statistics, users basically understood everything about karma for a long time.

Here is a record of a decade ago:

The big problem on the hub is that there are many users who put a minus in karma according to the principle: "Ah, you have an opinion that is different from mine, here's a minus in karma." Although, for me, a well-reasoned commentary with counterarguments and a well-stated opposite position does not deserve even a minus for the comment itself, not to the author’s karma. Unfortunately, on Habré there is practically no culture of reasoned disputes and respect for a strong opponent, many tend to just throw caps over their heads.

In general, I have an opinion that dividing ratings into two counters “rating” and “karma” is unintuitive and therefore incorrect and inefficient.

https://habr.com/en/post/92426/#comment_2800908

Here is a record five years ago:

Only cases where karma changed by at least 15 units were analyzed, but this does not change the picture as a whole, because and in this case, the ratio of 30% to 70%. Apparently, karma is merged for the most part because of comments, but raised for written articles.

https://habr.com/en/post/192376/

Here is a suggestion to improve three years ago:

Sentence:

Allow authors of articles to vote for karma only in a certain period (for example, a week) after they publish the article. If a person has not published anything in the last week, he cannot be leaked karma for comment. The rule can not be extended to reed-only accounts - they gain karma with useful comments.

Comment:

Too often, Habr users complain about karma drain for objectionable comments in other people's posts. For example, in this post the problem was described back in 2012. Woz and now there.

https://github.com/limonte/dear-habr/issues/49

Here is another dialogue three years ago on the same topic:

Drmetallius

I can say why I stopped writing comments (I will make this an exception): because it’s hard to earn karma, because for it you need to constantly generate some articles, but losing it is very easy. It is not true that if you write correctly, it is not wasted. It can be reduced for many reasons: I did not agree with you in the dispute, I considered that some fact was incorrect in the comment, or simply that he was in a bad mood.

maxshopen

Yes, this is an ancient disease of the habrasystem. It was assumed that those who have positive karma are adequate and just will not minus anyone. When everything was even worse - the more karma, the greater minus the user can bet that it ended up with a couple of “starred” habrausers handed out to the left and to the right who got -6, -8, after which the possibilities were cut to one. The creators of the karmic economy initially apparently did not take into account the perversity of anonymity

It seems to me that for a long time this system had to be balanced a little more by the fact that when voting to someone, a certain amount of it is debited from the user's karma. A lot is not necessary - 0.2-0.5 is enough. This would greatly increase the responsibility of voters when choosing to vote for someone or not.

https://habr.com/en/post/276383/#comment_8761911

And finally, comments on the post from the beginning of this year:

Karma is not a very good tool for system self-regulation. Those who are dissatisfied with a person (or even his position) are more likely to put in karma ratings. The result is that gaining karma is very difficult, and merging it is very easy. This makes people think again - is it worth expressing your opinion if it is not very popular? After all, if I express it once, they will kink and merge karma, and it will no longer be possible to express more. This leads to the fact that only one opinion remains on the resource, and all others are supplanted.

https://habr.com/en/company/habr/blog/437072/#comment_19647340

And here is a comment that explains why “writing articles” doesn’t really save the karma system:

The article brings almost nothing in terms of karma, and for one unsuccessful commentary, a person can be completely merged.

Here the problem is the separation of rating and karma. In the head of people it works like this:

1. Content rating is my attitude to the article or comment.

2. Evaluation of karma is my attitude to a person personally

Eventually

1. If you wrote the best article in the world, they will put you a lot of advantages to the article (in the rating) and consider your mission accomplished.

2. If you wrote a comment that “will not get into the stream” then you will have a minus comment, and even a person you probably think so-so because of this you and karma.

https://habr.com/en/company/habr/blog/437072/#comment_19649262

Many dissatisfied with the karma system express themselves in the sense that this is a deliberate policy of the administration - for example, in this commentary or in this . There are, of course, a lot of indirect evidence:

Removed the API so that it was not possible to monitor the dynamics;
We made a dynamic rating so that it was impossible to look at the total ratings directly in the profile;
They constantly refer to the “karmograph”, according to which there are more pluses than minuses (the ratio of karma and ratings is not even mentioned);
A lot, but unreasonably, they say that karma reflects the quality of publications and comments (which contradicts statistics, as we see from the evaluation indicators).

I also remind you that nowhere and never has the rationale for the existence of karma been given in the form in which it exists.

We cannot prove these conspiracy theories in any way. But it seems to me that the matter is not in them - there is the same problem as with people who are karma-negative: an impenetrable belief in their righteousness, to such an extent that those who disagree with you are perceived as a “bad person”. Here the Habr leaders in the same way decided - we will evaluate users separately from their messages. And they can’t be explained to them for more than ten years that this is an incorrect approach to ranking users. They are smart, they have created a whole portal. So you create your Habr - and then we’ll talk (by the way, it’s funny that literally in such terms the defender of karma claims to me and answered , “First get it”)

Personally, I assume that the karma scheme itself came to us from Lepra , where most of the current owners of large Internet portals were hanging out at one time. Habr began as the same Leprosy - a closed club with invites and mutual ratings, dissatisfied - went away from the club. Those days are long gone, the club has not been closed for a long time, ratings have long since been given not to “another member of the club”, but to an ordinary user for ordinary comments and articles. But internal elitism does not let go of the administration. Everyone thinks - indeed, the guys created a large profitable portal, write articles on technical topics for many years - can they not know something? So if everything is bad, then they, the villains, are conceived. But in fact, administrators are simply stuck in childhood. And the larger and more profitable the portal, the more difficult it is to admit many years of mistakes from falsely understood pride.

Confusion

These are deep waters, Watson, deep waters. I just started diving.

Special Issue of Sherlock Holmes

Below I will use the term “Karma” for karma, and the term “Evaluation” or “Total Evaluation” - for the total amount of all the pros and cons that the user received, both for articles and comments.

Having figured out the history, we will try to look at the numbers. Recently, there was a whole cycle of statistics analysis, but it concerned only the current year - I needed to understand the total user rating. Since we do not have an API, and instead of real ratings in the profile they show a dubious rating - I could only study each comment and collect data about the author and rating from it. That is what I did.

I opened every publication from the very beginning of time, took out the nickname of the author of the publication and the rating of the article, and then the nicknames of commentators and the ratings of their comments.

Here is the code for the main parser.

import requests
from bs4 import BeautifulSoup
import csv

def get_doc_by_id(pid):
    fname = r'files/' + 'habrbase' + '.csv'
    with open(fname, "a", newline="") as file:
        try:
            writer = csv.writer(file)
            r = requests.head('https://habr.com/ru/post/' +str(pid) + '/')
            if r.status_code == 404: #   
                pass
            else:
                r = requests.get('https://habr.com/ru/post/' +str(pid) + '/')
                soup = BeautifulSoup(r.text, 'html5lib')
                if not soup.find("span", {"class": "post__title-text"}):
                    pass
                else:
                    doc = []
                    cmt = []
                    doc.append(pid) #
                    doc.append(soup.find("span", {"class": "user-info__nickname"}).text) #
                    doc.append(soup.find("span", {"class": "voting-wjt__counter"}).text) #
                    writer.writerow(doc)
                    comments = soup.find_all("div", {"class": "comment"})
                    for x in comments:
                        if not x.find("div", {"class": "comment__message_banned"}):
                            cmt.append(x['id'][8:]) #
                            cmt.append(x.find("span", {"class": "user-info__nickname"}).text) #
                            cmt.append(x.find("span", {"class": "voting-wjt__counter"}).text) #
                            writer.writerow(cmt)
                            cmt = []
        except requests.exceptions.ConnectionError:
            pass

x = int(input())
y = int(input())

for i in range(x, y):
    get_doc_by_id(i)
    print(i)

habrbase:

« — » habrauthors.csv. . , - , , .

import requests
from bs4 import BeautifulSoup
import csv
import pandas as pd

def len_checker():
    fname = r'files/' + 'habrdata' + '.csv'
    with open(fname, "r") as file:
        try:
            authorsList = len(file.readlines())#   
        except:
            authorsList = 0
        return authorsList

def profile_check(nname):
    try:
        r = requests.head('https://m.habr.com/ru/users/' +nname + '/')
        if r.status_code == 404: #   
            pass
        else:
            ValUsers = []
            r = requests.get('https://m.habr.com/ru/users/' +nname + '/')
            soup = BeautifulSoup(r.text, 'html5lib') # instead of html.parser
            if not soup.find("div", {"class": "tm-user-card"}):
                valKarma = 0
                valComments = 0
                valArticles = 0
            else:
                valKarma = soup.find("span", {"class": "tm-votes-score"}).text #
                valKarma = valKarma.replace(',','.').strip()
                valKarma = float(valKarma)
                tempDataBlock = soup.find("div", {"class": "tm-tabs-list__scroll-area"}).text.replace('\n', '') # 
                mainDataBlock = tempDataBlock.split(' ')
                valArticles = mainDataBlock[mainDataBlock.index('')+1]
                if valArticles.isdigit() == True:
                    valArticles = int(valArticles)
                else:
                    valArticles = 0
                valComments = mainDataBlock[mainDataBlock.index('')+1]
                if valComments.isdigit() == True:
                    valComments = int(valComments)
                else:
                    valComments = 0
            ValUsers.append(valKarma)
            ValUsers.append(valComments)
            ValUsers.append(valArticles)
    except requests.exceptions.ConnectionError:
        ValUsers = [0,0,0]
    return ValUsers


def get_author_by_nick(x):
    finalRow = []
    df = pd.DataFrame
    colnames=['nick', 'scores']
    df = pd.read_csv(r'files\habrauthors.csv', encoding="ANSI", names = colnames, header = None)
    df1 = df.loc[x:]

    fname = r'files/' + 'habrdata' + '.csv'

    with open(fname, "a", newline="") as file:
        writer = csv.writer(file)
        for row in df1.itertuples(index=True, name='Pandas'):
            valName = getattr(row, "nick")
            valScore = getattr(row, "scores")
            valAll = profile_check(valName)
            finalRow.append(valName)
            finalRow.append(valScore)
            finalRow.append(valAll[0])
            finalRow.append(valAll[1])
            finalRow.append(valAll[2])
            writer.writerow(finalRow)
            print(valName)
            finalRow = []

n = len_checker()
get_author_by_nick(n)

, , - . , 2001 ? , , , : « (intermediate value).map is not a function». , , 2001 . , — . 250, — .

habrdata : ['nick', 'scores', 'karma', 'comments','articles','regdate']. .

. , .

	2006	2007	2008	2009	2010	2011	2012	2013	2014	2015	2016	2017	2018	2019
	2045	11668	12463	5028	5346	13686	11610	9614	9703	6594	8926	7825	5912	3673

114 343 , - . , :

, , tabloo.

, . , alizar (UPD) 268 ! , - 30 . — Zelenyikot 1509, - 500. , , .

-10

, , , , . , - . , , — CSV R, SPSS. , — , , ( ). 4235 . . 2866 , , .

3-4 114 , . , . 4652. , , .

: 114 343

< 5: 89 447

.. : 67 890

.. : 4 652

>= 5 : 24 896

, «». " ", . , . , « , » — .

:

: 114 343

<5: 57 223

.. : 26 207

.. : 9 737

>=5 : 57 120

, , , , . , , .. ! — , 90.

, , « », . — >=5 24 ( 900 - 5 ; ). 36 — .

, ? , , 90% , , 4 — . , « ». , , « ». . « , » .

, 2016 2018 , « ». 90% , 5 . « » 60-70% .

, :

78205 114 343 0. , .

24 896 114 343 . , , , .

. , , — , . , . , , « ».

     ,       !

, , , .

:

, .
1/3 , .
, ( 10-20 )

, , ?

, . , :

, , … :

— .

—

—

—

, , .

.

, — , .

https://habr.com/ru/post/467875/#comment_20639397

, — , .

,

, ?

, . , ( ).

.

1.

. : . , — : .

: , , , «» . — . . , . .

. , . — — . — . , , , — ; — , , , . , . — . — .

. . . . . , — .

. .

2.

.

— . , . , . , . , : , — , , . 2013 , , . ; , , , .

— , , . « », , , . — , . , , .

, «» «», . , , . — , .

, . , , — ? — , ? . , . , — , , .

— . , « », . , , . -, .

, , : , . .

— . , , . — (), , , . « » — — , .

, . . , — , .

, — . .

https://habr.com/ru/company/habr/blog/437072/#comment_19649328

, , — -, « ». , , .

.

3.

, . — , -, , .. . , .

— , . . — « », . , « »; , — , ( ) .

. , — , , , , . , , , , .

pragmatik,

, ., , - / — .

, , . , +100, -10? ? , , - ? .

. , . , , , . . «», « » , .

, . , . «-25», , . — , « ». , , . , , . ! .

« :

: «, , . , , , . — , . .»

, — . andorro : « ».»

https://habr.com/ru/company/habr/blog/437072/#comment_19822200

, . — (.. ). « ». — , . « », . , , IRC, , « » « ». — , .

Episode Three

— ?

— , ,

« »

?

-, , , , . , , . — , , , , . , , , .

-, . 10 20 , ? 118, - 100, -100 , , , 10.

-, , , (.. ). « » — , : , .

-, , .. « » — - . , , - « ». , ; , — .

-, , « ». . — , — . .

— , . - « , , » - .

All Articles

Habra's karmic curse

Unforeseen Consequences

Confusion

,

Episode Three

More articles: